System and method for generating search and extended patches for automated program repair.

Abstract

A system and method for automatic program repair using a neural network model will be described. After a first bug-containing code patch is received, a first representation of the first bug-containing code patch is generated using a retriever encoder of a patch retriever. The patch retriever searches for a first bug-fixing code pair from a first plurality of bug-fixing code pairs based on the first representation. A first extended bug-containing code patch is generated based on the first bug-containing code patch and the first bug-fixing code pair. A patch generator generates a repaired code patch based on the first extended bug-containing code patch.

Description

[Technical Field]

[0001] [Cross reference] This disclosure claims priority to U.S. Nonprovisional Patent Application No. 17 / 896,873, filed on 26 August 2022, which in turn claims priority to U.S. Provisional Patent Application No. 63 / 343,264, filed on 18 May 2022 under Section 119 of the U.S. Patent Act, both of which are incorporated herein by reference in their entirety.

[0002] [Technical field] The embodiments generally relate to machine learning and automated code generation, and more specifically to systems and methods for automated program repair (APR) using Search and Enhanced Patch Generation (RAP-Gen). [Background technology]

[0003] Software developers often spend a significant amount of time and energy debugging and repairing source code, making software development expensive and time-consuming. Some existing automated program repair tools reduce the difficulty and cost of program repair in use cases that include patch searching during development, build, or runtime. For example, some search-based (also called generate-and-validate) approaches may search for repairs based on correction patterns mined through manual heuristic rules or redundancy-based techniques. Redundancy-based techniques generally make the redundancy assumption that corrected patches can often be found (or reconstructed) elsewhere in the codebase (donor code snippets). Consequently, these traditional search-based techniques have limited accuracy and efficiency when it comes to repairing programs.

[0004] Therefore, a more efficient method for automated program repair is needed. [Brief explanation of the drawing]

[0005] [Figure 1] A simplified diagram of a computing device for implementing the automatic program repair framework described in FIG. 3 and other embodiments described herein. [Figure 2] A simplified block diagram of a networked system suitable for implementing the automatic program repair framework described in FIG. 3 and other embodiments described herein. [Figure 3] An exemplary block diagram showing an exemplary architecture for an automatic program repair framework using search extension patch generation according to some embodiments described herein. [Figure 4A] An exemplary logic flow diagram showing a method for training a search extension patch generation framework for automatic program repair as shown in FIG. 3 according to some embodiments described herein. [Figure 4B] An exemplary logic flow diagram showing a method for an inference process using a trained search extension patch generation framework according to some embodiments described herein. [Figure 5] Provide an exemplary data table showing exemplary data performance of an automatic program repair framework using search extension patch generation as described in connection with FIGS. 1 - 4B according to some embodiments described herein. [Figure 6] Provide an exemplary data table showing exemplary data performance of an automatic program repair framework using search extension patch generation as described in connection with FIGS. 1 - 4B according to some embodiments described herein. [Figure 7] Provide an exemplary data table showing exemplary data performance of an automatic program repair framework using search extension patch generation as described in connection with FIGS. 1 - 4B according to some embodiments described herein. [Figure 8]This specification provides exemplary data tables illustrating the data performance of an automated program repair framework using search-enhanced patch generation, as described in relation to Figures 1-4B, according to several embodiments described herein. [Figure 9] This specification provides exemplary data tables illustrating the data performance of an automated program repair framework using search-enhanced patch generation, as described in relation to Figures 1-4B, according to several embodiments described herein. [Figure 10] This specification provides exemplary data tables illustrating the data performance of an automated program repair framework using search-enhanced patch generation, as described in relation to Figures 1-4B, according to several embodiments described herein. [Figure 11] This specification provides exemplary data tables illustrating the data performance of an automated program repair framework using search-enhanced patch generation, as described in relation to Figures 1-4B, according to several embodiments described herein. [Figure 12] This specification provides exemplary data tables illustrating the data performance of an automated program repair framework using search-enhanced patch generation, as described in relation to Figures 1-4B, according to several embodiments described herein. [Figure 13] This specification provides exemplary data tables illustrating the data performance of an automated program repair framework using search-enhanced patch generation, as described in relation to Figures 1-4B, according to several embodiments described herein. [Figure 14] This specification provides exemplary data tables illustrating the data performance of an automated program repair framework using search-enhanced patch generation, as described in relation to Figures 1-4B, according to several embodiments described herein. [Figure 15] This specification provides exemplary data tables illustrating the data performance of an automated program repair framework using search-enhanced patch generation, as described in relation to Figures 1-4B, according to several embodiments described herein.

[0006] In these diagrams, elements with the same sign have the same or similar function. [Modes for carrying out the invention]

[0007] As used herein, the term "Network" may include any hardware or software-based framework, including any artificial intelligence network or system, a neural network or system, and / or any training or learning model implemented on or with therein.

[0008] As used herein, the term “module” may include a hardware or software-based framework that performs one or more functions. In some embodiments, a module may be implemented on one or more neural networks.

[0009] Existing automated program repair systems can reduce manual debugging and improve software reliability. Traditional search-based techniques typically rely on heuristic rules or redundancy assumptions to mine repair patterns. Some deep learning-based approaches can automate the program repair process by training a learning model to generate code repair patches. However, the performance of such learning models is often limited by a fixed set of parameters for modeling the highly complex search space of program repair.

[0010] In light of the need for an efficient and accurate code repair system, the embodiments described herein provide a search-enhanced patch generation framework for retrieving code patches using a patch retriever based on relevant fix patterns. Specifically, a hybrid patch retriever can be configured for fix pattern mining that considers both lexical and semantic matching through sparse and dense searches based on raw source code. Furthermore, this retriever is language-independent as it does not require language-specific features such as abstract syntax trees. One improvement over previous fix pattern mining models is that the retriever utilizes a top-tier relevant bug-fix pair as a guide fix pattern for each buggy patch, rather than clustering various fix templates. This strategy aligns with the debugging behavior of human developers, who often explore relevant bug fix examples to extract several fix cues for bug fixes.

[0011] In one embodiment, a pre-trained Transformer-based encoder-decoder model (e.g., the CodeT5 model) may be employed as the underlying patch generator. CodeT5 is a generic programming language model pre-trained on a large source code corpus using the Code Recognition Language Modeling Goal. A two-stage training strategy may be used to train the pre-trained encoder-decoder model to connect the patch retriever with the CodeT5 patch generator. The patch retriever first searches for relevant bug fix patterns and then passes them to the patch generator to synthesize corrected patches based on both the source bug-containing code (buggy code) and external (searched) bug fix knowledge. The searched fix patterns can then be directly added to the source bug-containing patches. In this way, the retriever can be integrated with any sequence-to-sequence learning-based model for searching in fix pattern mining for program repair.

[0012] Figure 1 is a simplified diagram of a computing device 100 for implementing the automated program repair framework shown in Figure 3, according to several embodiments. As shown in Figure 1, the computing device 100 includes a processor 110 coupled to memory 120. The operation of the computing device 100 is controlled by the processor 110. Although the computing device 100 is shown with only one processor 110, it is understood that the processor 110 may represent one or more central processing units, multicore processors, microprocessors, microcontrollers, digital signal processors, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), graphics processing units (GPUs), etc., within the computing device 100. The computing device 100 may be implemented as a standalone subsystem, as a board added to a computing device, and / or as a virtual machine.

[0013] Memory 120 may be used to store software executed by computing device 100 and / or one or more data structures used during the operation of computing device 100. Memory 120 may include one or more types of machine-readable media. Some common forms of machine-readable media may include floppy disks, flexible disks, hard disks, magnetic tapes, any other magnetic media, CD-ROMs, any other optical media, punch cards, paper tapes, any other physical media having a pattern of holes, RAM, PROMs, EPROMs, FLASH-EPROMs, any other memory chips or cartridges, and / or any other media adapted to be read by a processor or computer.

[0014] The processor 110 and / or memory 120 can be located in any suitable physical arrangement. In some embodiments, the processor 110 and / or memory 120 may be implemented on the same board, in the same package (e.g., system-in-package), on the same chip (e.g., system-on-chip), etc. In some embodiments, the processor 110 and / or memory 120 may include distributed, virtualized, and / or containerized computing resources. Consistent with such embodiments, the processor 110 and / or memory 120 may be located in one or more data centers and / or cloud computing facilities.

[0015] In some examples, memory 120 may include a non-temporary tangible machine-readable medium containing executable code that, when executed by one or more processors (e.g., processor 110), can cause one or more processors to execute methods further described herein. For example, as shown in the figure, memory 120 includes instructions for an automated program repair module 130 that can be used to implement and / or emulate a system and model, and / or to implement any of the methods further described herein. The automated program repair module 130 may receive inputs 140, including program bugs, via a data interface 115. The automated program repair module 130 may generate outputs 150, such as code patches.

[0016] In some embodiments, the automatic program repair module 130 includes a retriever encoder submodule 131, a patch retriever submodule 132, and a patch generator submodule 133. In one embodiment, the automatic program repair module 130 and its submodules 131-133 may be implemented by hardware, software, and / or a combination thereof.

[0017] Some examples of computing devices, such as computing device 200, may include non-temporary, tangible, machine-readable media containing executable code that, when executed by one or more processors (e.g., processor 110), causes one or more processors to execute the process of the Method. Some common forms of machine-readable media that may contain the process of the Method include, for example, floppy disks, flexible disks, hard disks, magnetic tapes, any other magnetic media, CD-ROMs, any other optical media, punch cards, paper tapes, any other physical media having a pattern of holes, RAM, PROMs, EPROMs, FLASH-EPROMs, any other memory chips or cartridges, and / or any other media adapted to be read by a processor or computer.

[0018] Figure 2 is a simplified block diagram of a networked system suitable for implementing the automated program repair framework described in Figure 3 and other embodiments described herein. In one embodiment, block diagram 200 shows a system including a user device 210 which can be operated by a user 240, data vendor servers 245, 270, and 280, a server 230, and other forms of devices, servers, and / or software components that operate to perform various methodologies according to the embodiments described. Illustrative devices and servers may include devices similar to the computing device 100 described in Figure 1, standalone, and enterprise-class servers that run an OS such as MICROSOFT® OS, UNIX® OS, LINUX® OS, or other suitable device and / or server-based OS. It can be understood that the devices and / or servers shown in Figure 2 may be deployed in other ways, and the operations performed, and / or services provided by such devices and / or servers, may be combined or separated for a given embodiment, and may be performed by more or fewer devices and / or servers. One or more devices and / or servers may be operated and / or maintained by the same or different entities.

[0019] The user device 210, data vendor servers 245, 270, and 280, and server 230 can communicate with each other via network 260. The user device 210 may be used by a user 240 (e.g., a driver, system administrator, etc.) to access various functions available to the user device 210, which may include processes and / or applications associated with server 230 for receiving output data anomaly reports.

[0020] The user device 210, the data vendor server 245, and the server 230 may each include one or more processors, memories, and other suitable components for executing instructions, such as program code and / or data, stored on one or more computer-readable media, in order to implement the various applications, data, and steps described herein. For example, such instructions may be stored on one or more computer-readable media, such as memory or data storage devices, that are internal to and / or external to the various components of the system 200, and / or accessible via the network 260.

[0021] The user device 210 may be implemented as a communication device that can utilize appropriate hardware and software configured for wired and / or wireless communication with the data vendor server 245 and / or server 230. For example, in one embodiment, the user device 210 may be implemented as other types of computing devices capable of transmitting and / or receiving data, such as an autonomous vehicle, a personal computer (PC), a smartphone, a laptop / tablet computer, a wristwatch with appropriate computer hardware resources, eyeglasses with appropriate computer hardware (e.g., GOOGLE GLASS®), other types of wearable computing devices, an embedded communication device, and / or an Apple® iPad®. Although only one communication device is shown, multiple communication devices may function similarly.

[0022] The user device 210 in Figure 2 includes a user interface (UI) application 212 and / or other applications 216, which may correspond to applications having executable processes, procedures, and / or associated hardware. For example, the user device 210 may receive a message from the server 230 indicating bug-containing code and / or corrected code, and display the message via the UI application 212. In other embodiments, the user device 210 may include additional or different modules having dedicated hardware and / or software, as needed.

[0023] In various embodiments, the user device 210 may include other applications 216 that are desired in certain embodiments to provide functionality to the user device 210. For example, other applications 216 may include security applications for implementing client-side security functions, program client applications for interfaceing with appropriate application programming interfaces (APIs) over the network 260, or other types of applications. Other applications 216 may also include communication applications such as email, texting, voice, social networking, and IM applications that enable the user to send and receive emails, calls, texts, and other notifications over the network 260. For example, other applications 216 may be email or instant messaging applications that receive predictive result messages from the server 230. Other applications 216 may include device interfaces and other display modules that can receive input and / or output information. For example, other applications 216 may include processor-executable asset management software programs that include a graphical user interface (GUI) configured to provide the user 240 with an interface for viewing bug-containing code and / or corrected code.

[0024] The user device 210 may further include a database 218 stored in the temporary and / or non-temporary memory of the user device 210, which stores various applications and data and can be used while various modules of the user device 210 are running. The database 218 may store a user profile of the user 240, predictions previously viewed or saved by the user 240, historical data received from the server 230, and so on. In some embodiments, the database 218 may be local to the user device 210. However, in other embodiments, the database 218 may be external to the user device 210 and accessible by the user device 210, which may include a cloud storage system and / or database accessible via the network 260.

[0025] The user device 210 includes at least one network interface component 219 adapted to communicate with the data vendor server 245 and / or server 230. In various embodiments, the network interface component 219 may include a DSL (e.g., Digital Subscriber Line) modem, a PSTN (Public Switched Telephone Network) modem, an Ethernet® device, a broadband device, a satellite device, and / or various other types of wired and / or wireless network communication devices, including microwave, radio frequency, infrared, Bluetooth®, and short-range communication devices.

[0026] The data vendor server 245 may correspond to a server hosting one or more of the databases 203a-n (or collectively referred to as 203) in order to provide the server 230 with training datasets containing pairs of bug-containing code and corrected code. Database 203 may be implemented by one or more relational databases, distributed databases, cloud databases, etc.

[0027] The data vendor server 245 includes at least one network interface component 226 adapted to communicate with the user device 210 and / or the server 230. In various embodiments, the network interface component 226 may include a DSL (e.g., Digital Subscriber Line) modem, a PSTN (Public Switched Telephone Network) modem, an Ethernet® device, a broadband device, a satellite device, and / or various other types of wired and / or wireless network communication devices, including microwave, radio frequency, infrared, Bluetooth®, and short-range communication devices. For example, in one implementation, the data vendor server 245 may transmit asset information from the database 203 to the server 230 via the network interface 226.

[0028] Server 230 may house the automatic program repair module 130 and its submodules as shown in Figure 1. In some implementations, module 130 may receive data from database 219 at data vendor server 245 via network 260 to generate corrected code patches. The generated corrected code patches may be sent via network 260 to user device 210 for review by user 240.

[0029] The database 232 may be stored in the temporary and / or non-temporary memory of the server 230. In one implementation, the database 232 may store data obtained from the data vendor server 245. In one implementation, the database 232 may store parameters of the automatic program repair module 130. In one implementation, the database 232 may store modified patches of previously generated code and corresponding input feature vectors.

[0030] In some embodiments, the database 232 may be local to the server 230. However, in other embodiments, the database 232 may be external to the server 230 and accessible by the server 230, which includes a cloud storage system and / or database accessible via the network 260.

[0031] Server 230 includes at least one network interface component 233 adapted to communicate with user devices 210 and / or data vendor servers 245, 270, or 280 via network 260. In various embodiments, the network interface component 233 may include various other types of wired and / or wireless network communication devices, including DSL (e.g., Digital Subscriber Line) modems, PSTN (Public Switched Telephone Network) modems, Ethernet® devices, broadband devices, satellite devices, and / or microwave, radio frequency (RF), and infrared (IR) communication devices.

[0032] Network 260 can be implemented as a single network or a combination of multiple networks. For example, in various embodiments, network 260 may include the Internet or one or more intranets, terrestrial communication line networks, wireless networks, and / or other suitable types of networks. Thus, network 260 can correspond to small-scale communication networks such as private or local area networks, or large-scale networks such as wide area networks or the Internet, accessible by various components of system 200.

[0033] FIG. 3 is an exemplary block diagram showing an exemplary architecture of an automatic program repair framework 300, also referred to as the RAP-Gen framework 300, according to the embodiments described herein. The RAP-Gen framework 300 aims to generate a target program patch based on an input bug-containing patch, along with related bug-fixing patterns through search.

[0034] The task formulation of search-augmented patch generation for automatic program repair is described as follows.

Number

[0035] , Y i ), where X i and Y i are the i-th bug-containing program patch and the corrected program patch, respectively. The codebase C (e.g., codebase 302) contains a large set of previous bug-fixing pairs

Number

number

number

number

number

number

[0036] As shown in the example in Figure 3, the RAP-Gen framework 300 includes a patch retriever 304 and a code recognition pre-trained patch generator 306. The patch retriever 304 is configured to search for relevant correction patterns that help in automated program repair. This is done using a relevance scoring function f φ (X i B j Based on ), (query) bug-containing patch Xi308 and the previous (key) bug-containing patch B in codebase C302 j The relationship between them can be calculated. In various embodiments, the patch retriever 304 may include a lexical-based retriever (e.g., BM25) and / or a semantic-based retriever (e.g., Dense Passage Retrieval (DPR)). In the example in Figure 3, the patch retriever 304 includes a neural network model (e.g., retriever encoder 318) and uses a hybrid approach to combine a lexical-based retriever (e.g., BM25) and a semantic-based retriever (e.g., DPR) to take both lexical and semantic information into consideration.

[0037] Lexical-based Retriever. In some embodiments, a lexical-based retriever (e.g., BM25) may be implemented using a term-based retriever, which may use sparse vector representations for lexical matching. The lexical-based retriever transforms each code patch as a bag-of-words representation and queries the patch X i and candidate patch B j The lexical similarity between them can be calculated. The calculated similarity score is f φ (X i ,B j )=BM25(X i ,B jThis is expressed as follows: For example, a sparse term base retriever may be sensitive to the choice of identifier naming in source code that does not affect the semantics of the code.

[0038] Semantic-based Retriever. In some embodiments, a semantic-based retriever may be implemented using a Dense Passage Retriever (DPR) that can retrieve relevant patches by measuring their semantic similarity. In some embodiments, an encoder (e.g., a Transformer-based encoder) may be used to encode code patches, mapping each patch to a fixed-size dense vector. The DPR may be initialized from an encoder of a pre-trained Transformer-based neural network model (e.g., Code Bidirectional Encoder Representations from Transformers (CodeBERT)). The encoder may be pre-trained using a large code repository of one or more programming languages ​​(e.g., a GitHub code repository for six programming languages). In one example, the hidden state of the final layer of the [CLS] token from the encoder is used as the patch representation. In some embodiments, a shared DPR is used to query patch X i Candidate patch B in 308 and C j Each of them,

number

number

[0039] In some embodiments, shared DPR is used to query patch X iCandidate patch F in 308 and C j Each of them,

number

number

[0040] The descriptions in this specification are generally for searching X i and B j Similarity between (for example, f φ (X i ,B j ) uses ) but the similarity used for the search is X i and B j Similarity between (for example, f φ (X i ,B j ) use), X i and F j Similarity between (for example, f φ (X i ,F j Please note that this may include using, and / or combinations thereof.

[0041] In some embodiments, a semantic-based retriever (e.g., DPR) is further trained using a training dataset containing pairs of bug-containing patches and corrected patches. In one example, bug-containing code B j Treat this as a query and apply the corresponding modified code F j By treating the key, a codebase 302 containing bug fix pairs can be used. This can be done on the assumption that bug-containing patches and their corrected patches often share similar semantics (e.g., identifiers, data flow, and code structure). This technique can be used to avoid the enormous amount of manual annotation work required to curate bug-to-bug search datasets.

[0042] In an example where bug fix pairs are used as queries and corresponding keys, a contrast learning method 314 with intra-batch negatives is used to train a semantic-based retriever, where intra-batch negatives are used to optimize the contrast loss (e.g., InfoNCE contrast loss) as follows:

number

[0043] In some embodiments, during the inference stage, query bug-containing patch X i Given 308, a semantic-based retriever (e.g., DPR) is X i (Query) and B j By calculating the similarity between (key) and (B), the associated bug fix pair (B) is determined. j ,F j ) searches. In some embodiments, the semantic-based retriever searches X i and F j Similarity between and / or X i (Query) and B j Based on the combination of similarities with (key), related bug fix pairs can be searched.

[0044] Hybrid Retriever. As shown in the example in Figure 3, in some embodiments, a hybrid approach is used that combines a lexical retriever (e.g., BM25) and a semantic retriever (e.g., DPR) to take both lexical and semantic information into consideration. For example, similarity scores are:

number

number

[0045] In the example in Figure 3, the RAP-Gen framework 300 includes a patch generator 306 for generating corrected code patches. In some embodiments, the code recognition pre-training patch generator 306 generates input bug-containing patches 308 (X), also called source bug-containing patches 308 or query bug-containing patches 308. i (as indicated), and the searched bug fix pattern 310 (B j F j Using (as shown), for example, the following concatenation is used to generate the extended bug-containing code patch 312:

number

[0046] In some embodiments, the patch generator 306 includes a code-aware programming language model pre-trained on a large source code corpus. For example, the sequence generator uses CodeT5, an integrated, pre-trained Transformer-based encoder-decoder model that achieves state-of-the-art (SoTA) results in multiple code intelligence tasks, including defect detection and code refinement. It can be pre-trained on 8.3 million functions in eight different programming languages ​​(including JavaScript® and Java®) collected from GitHub. CodeT5 may incorporate code-specific knowledge into the language model by employing identifier-aware pre-training objectives. This may provide a code-specific byte-pair coding (BPE) tokenizer optimized for the code, potentially avoiding out-of-vocabulary (OoV) problems. CodeT5 can be used in the patch generator 306, which may provide robust code understanding capabilities.

[0047] As shown in the example in Figure 3, the search extension input 312 to the patch generator 306 (e.g., CodeT5) is:

number

number

[0048] In various embodiments, the RAP-Gen framework 300 leverages general code understanding knowledge encoded through pre-training on a large code corpus (e.g., using CodeT5). For example, a source input sequence 312 may be generated by concatenating the original bug-containing code patch 308 with top-ranked bug fix pairs 310 from a patch retriever 304. In some embodiments, an extended source input bug-containing patch 312 may be generated by concatenating top-k (e.g., k=2,3,5) retrieved bug fix pairs to the input bug-containing patch 308.

[0049] Figure 4A is an exemplary logic flow diagram illustrating how to train a search-extended patch generation framework for automated program repair, as shown in Figure 3, according to some embodiments described herein. One or more of the processes of Method 400 may be implemented in the form of executable code stored on a non-temporary tangible machine-readable medium, which, when executed by one or more processors, can cause one or more processors to execute one or more of the processes. In some embodiments, Method 400 corresponds to the operation of an automated program repair module 130 (e.g., Figure 1) for performing automated program repair using search-extended patch generation.

[0050] In step 402, a patch retriever including a retriever encoder is provided. In the example in Figure 3, the search-enhanced patch generation framework 300 provides a patch retriever 304 including a retriever encoder 318. In some embodiments, as shown in step 404, the retriever encoder 318 is pre-trained using a first training dataset, for example, including a large programming language corpus (e.g., a GitHub code repository or other suitable code repository in one or more programming languages).

[0051] In step 406, a patch generator is provided that includes a sequence generator neural network model. In the example in Figure 3, in the search-enhanced patch generation framework 300, the patch generator 306 includes a sequence generator neural network model, specifically a Transformer-based encoder-decoder model that includes a generator encoder 318 and a generator decoder 320. In some embodiments, as shown in step 404, the patch generator 306 is pre-trained using a second training dataset that includes, for example, a large programming language corpus (e.g., a GitHub code repository or other suitable code repository in one or more programming languages).

[0052] In step 410, the RAP-Gen framework, including the patch retriever and patch generator (e.g., the RAP-Gen framework in Figure 3), may be trained using, for example, a two-stage training process. The two-stage training process includes step 412, in which the first stage training is performed by training the patch retriever using a third training dataset. In some embodiments, the third training dataset may be bug fix pairs within codebase 302. For example, when using the third training dataset to train the semantic retriever of patch retriever 304, the bug-containing code Bj of the bug fix pair in the codebase may be considered a query, and the corresponding fixed code Fj may be considered a key. In another example, the fixed code Fj of the bug fix pair in the codebase may be considered a query, and the corresponding bug-containing code Bj may be considered a key. This is based on the assumption that bug-containing patches and their fixed patches often share similar semantics (e.g., identifiers, data flow, and code structure). By using bug fix pairs from codebase 302 for the third training dataset, the extensive manual annotation work required to curate the bug-to-bug search dataset as the third training dataset can be avoided. For example, the first stage of training could use a symmetric learning algorithm by optimizing the symmetric loss.

[0053] The two-stage training process includes step 414, in which the second stage training is performed by training a patch generator using a fourth training dataset with the patch retriever trained in the first stage training. For example, if the input to the patch generator is generated using the original input bug-containing code patch and top-ranked bug fix pairs from the trained patch retriever, a supervised enforcement algorithm is used to minimize language modeling loss.

[0054] During the second stage of training, in an example where the fourth training set is generated from a bug fix pair codebase, the patch retriever (already trained using the first stage of training) is not allowed to access the ground truth bug fix pairs. Otherwise, the training loss would easily drop to near zero, as the patch generator could directly copy the retrieved fixes as the target output. In that example, each sample in the fourth training set is a bug-containing patch of the corresponding bug fix pair (also called a ground truth bug fix pair) from the codebase, and the corresponding ground truth is the corrected patch of the corresponding bug fix pair. For each sample bug-containing patch input, another bug fix pair (not from ground truth) is retrieved from the codebase by the patch retriever. The retrieved bug fix pair is appended to the bug-containing patch input to generate an extended sequence input for the patch generator. Please note that the requirement of not having access to Ground Truth bug fix pairs applies only to the second stage of training when the codebase is used to provide the fourth training set, and not to the first stage of training the patch retriever when the codebase is used to provide the third training set.

[0055] How are the third and fourth datasets generated? Recall that there is a bug fix pair for each downstream dataset, which is precisely the third training set.

[0056] Referring to Figure 4B, an exemplary logic flow diagram is shown illustrating Method 450 of an inference process using a trained search-extended patch generation framework, according to some embodiments described herein. One or more of the processes of Method 450 may be implemented, at least in part, in the form of executable code stored on a non-temporary tangible machine-readable medium, which, when executed by one or more processors, can cause one or more processors to execute one or more of the processes. In some embodiments, Method 450 corresponds to the operation of an automated program repair module 130 (e.g., Figure 1) for performing automated program repair using search-extended patch generation.

[0057] In step 452, the trained search-enhanced patch generation framework receives the first bug-containing patch. In the example in Figure 3, the trained search-enhanced patch generation framework 300 receives the first bug-containing patch 308 and provides it as input to its trained patch retriever 304.

[0058] In step 454, one or more bug fix pairs are provided based on the first bug-containing patch. In the example in Figure 3, the trained patch retriever 304 receives the first bug-containing patch 308 and retrieves one or more bug fix pairs from the codebase 302 based, for example, on the similarity between the first bug-containing patch 308 and the bug fix pairs. In various embodiments, the similarity is determined based on the similarity between the first bug-containing patch 308 and the bug-containing patch of the bug fix pair, the similarity between the first bug-containing patch 308 and the corrected patch of the bug fix pair, or a combination thereof. The similarity may include lexical similarity, semantic similarity, or a combination thereof.

[0059] In step 456, a first extended bug-containing patch is generated based on the first bug-containing patch and one or more bug fix pairs found. In the example in Figure 3, the first extended bug-containing patch 312 is generated using the first bug-containing patch 308 and one or more bug fix pairs 310 provided by the patch retriever 304. The first extended bug-containing patch 312 is provided to the patch generator 306.

[0060] In step 458, the first extended bug-containing patch is used to generate a first corrected patch for the first bug-containing patch. In the example in Figure 3, the patch generator 306 receives the first extended bug-containing patch 312 and generates a first corrected patch 316 based on the first extended bug-containing patch 312. Exemplary data experiments and performance

[0061] Referring to Figure 5, in several experiments, the RAP-Gen framework has been evaluated on two common APR datasets: TFix in JavaScript (Berkay Berabi, Jingxuan He, Veselin Raychev, and Martin T. Vechev, TFix: Learning to Fix Coding Errors with a Text-to-Text Transformer, Proceedings of Machine Learning Research (PMLR), Vol. 139, 780-791) and Code Refinement in Java (Michele Tufano, Cody Watson, Gabriele Bavota, Massimiliano Di Penta, Martin White, and Denys Poshyvanyk, An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translation, ACM Trans. Softw. Eng. Methodol. 28, 4 (2019), 19:1-19:29). Both datasets are originally collected from GitHub commits, but there is a significant difference: bug fix pairs in TFix can be validated by a static analyzer, while pairs in Code Refinement are validated by checking whether the commit message contains keywords such as "fix bug". Data statistics for the TFix and Code Refinement benchmarks are shown in Table 1 of Figure 5.

[0062] TFix. Specifically, TFix is ​​a large program repair dataset containing JavaScript code patch pairs curated from 5.5 million GitHub commits. It comprehensively covers 52 unique error types detected by the static analyzer ESLint. In addition to error types, it provides rich error annotations such as error messages and localized error lines, eliminating the need for fault localization as in traditional methods. TFix tackles the APR task as a text-to-text generation problem using T5-large. In the source input sequence, they combine all error information along with the bug-containing code patch into a single text: fix {error type} {error message} {error context} Here, the error context consists of a given localized error line, and its two adjacent lines of code are used to form a bug-containing code patch. The target sequence is to replace the error line with a corrected line in the error context. The same data format is used in the experiment, and an example of the data can be found in the source input in Figure 6 (showing one example of bug correction in the TFix test set where the RAP-Gen framework correctly fixes the bug).

[0063] During data processing, duplication issues were observed within and between data splits. Specifically, there were 114, 2, and 4 duplicates in the training, validation, and test splits, respectively. Regarding duplication between splits, there were 28, 34, and 4 duplicates between the training and test splits, the training and test splits, and the validation and test splits, respectively. After filtering out these duplicates (243), the deduplication-free version TFix (Dedup) is shown in Table 1 of Figure 5.

[0064] Code Refinement. Tufano et al. released two code refinement datasets containing bug fix pairs at the function level, collected from the GitHub Archive (https: / / www.gharchive.org / ) published between March 2011 and October 2017. To ensure the quality of the collected bug fix function pairs, the Google BigQuery API was used to identify all Java commits with messages containing the patterns ("fix" or "solve") and ("bug", "issue", "problem", or "error"). Functions were normalized by obfuscating identifiers using indexed tokens such as TYPE1, VAR1, and METHOD1. One example of the data can be seen in Figure 7 (showing one example of a bug fix in the Refinement Small test set, where the RAP-Gen framework makes the correct prediction). The two data subsets are determined by the number of tokens: for the small set, the number of code tokens ≤ 50, and for the medium set, 50 < number of code tokens ≤ 100.

[0065] In some embodiments, the RAP-Gen framework 300 may be fine-tuned for each benchmark using sequence-to-sequence generation loss (e.g., over 30 epochs), for example, using the AdamW optimizer (Ilya Loshchilov and Frank Hutter DecoupledWeight Decay Regularization, ICLR, 2019). Grid search may be performed for hyperparameter tuning using various batch sizes (e.g., 16, 32, 64) and learning rates (e.g., 1e-4, 5e-5, 2e-5). For example, a learning rate of 1e-4 and a batch size of 64 may be used for TFix, and a learning rate of 5e-5 and a batch size of 32 may be used for Code Refinement. In one example, the training time for RAP-Gen-base in each benchmark using one A100 GPU is less than two days. During inference, beam search may be employed with a beam size of 5 to generate a ranked list of synthesized modified patches.

[0066] In some embodiments, bug fix pairs within the training set are used as the search codebase for building the patch retriever 304. For vocabulary-based retrievers, exemplary open-source Python libraries (e.g., BM25 at https: / / pypi.org / project / rank-bm25) may be used. As a sparse term-based retriever, the choice of tokenizer significantly impacts search performance. In the experiment, the CodeT5 tokenizer, a code-specific BPE tokenizer optimized for code tokenization, is employed. The BM25 search engine for benchmarks TFix and Code Refinement is applied on a 95-CPU machine with 600GB of memory. Each experiment is completed within one hour with multiple processing.

[0067] In the experiment, for the semantic-based retriever, a CodeBERT initialized with DPR was used to encode each patch into a dense vector for semantic matching. The DPR model was then fine-tuned for each benchmark over 50 epochs using InfoNCE controlled loss. A batch size of 64 and a learning rate of 2e-5 were used for fine-tuning on a single A100 GPU with 40GB of memory. Training times for TFix and Code Refinement were approximately 9 and 5 GPU hours, respectively.

[0068] For hybrid retrievers, BM25 and DPR ranking scores are calculated, and these normalized scores are linearly combined with equal weights to construct the hybrid retriever, i.e., "Hybrid". For all retrievers, a CodeT5 tokenizer is used to encode patches with a maximum sequence length of 256.

[0069] Evaluation Metrics. For evaluation metrics, program repair performance is assessed using smoothed BLEU-4 (Chin-Yew Lin and Franz Josef Och, ORANGE: a Method for Evaluating Automatic Evaluation Metrics for Machine Translation, COLING, 2004) scores and perfect match (EM) accuracy (e.g., following Yue Wang, Weishi Wang, Shafiq R. Joty, and Steven CH Hoi, CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation, EMNLP, Association for Computational Linguistics, 8696-8708). BLEU-4 is a looser metric for assessing the degree of subword overlap, while EM is a stricter metric requiring that the prediction be identical to the ground truth patch in the actual commit. Since bug-containing programs may have different repair methods, various forms of correction are considered using Error Removal metrics (e.g., as used in TFix). If existing errors are removed and no new errors are introduced after the fix, the prediction for Error Removal is counted as correct. For all metrics, the results are presented on a scale of 0 to 100%, with higher scores indicating better performance.

[0070] Baseline Models. The RAP-Gen framework is compared to learning-based models in two program repair benchmarks. CoCoNuT is a context-aware neural machine translation framework based on a convolutional encoder-decoder model. SequenceR is an LSTM-based sequence-to-sequence generation model with a copy mechanism. In addition, the RAP-Gen framework is compared to pre-trained programming language models based on the Transformer architecture. One group of these models is encoder-only models such as RoBERTa(Code), CodeBERT, and GraphCodeBERT. These encoder-only models require a randomly initialized decoder for the program repair task.

[0071] Furthermore, the RAP-Gen framework is compared to an encoder-decoder Transformer model. PLBART is an integrated pre-trained model with denoising objectives including token masking, token deletion, and token infilling. TFix is ​​initialized with a T5-large checkpoint and continues to be fine-tuned on the TFix dataset. CoTexT is another T5-based model pre-trained on both text and code. NSEdit is a language model with the encoder and decoder initialized from CodeBERT and CodeGPT, respectively. It is fine-tuned to generate modifications via a neural symbol editing sequence and ranks as the current SoTA model on the Code Refinement benchmark. Results from all baseline models are taken from their original papers.

[0072] The experiment verifies that search-enhanced patch generation is an effective approach for program repair. A comprehensive experiment was conducted to compare RAP-Gen with conventional learning-based methods on two benchmarks. First, the CodeT5 model was evaluated against TFix, and its evaluation was improved by providing a deduplication version of the dataset and more rational metrics, as well as by additionally introducing a looser metric for exact matches and matching BLEU-4 scores. As a result, CodeT5-base established new state-of-the-art performance on this task, improving EM from 49.70 to 53.57 compared to T5-large, and BLEU-4 from 76.98 to 78.85. Furthermore, the RAP-Gen model was evaluated using both the TFix and Code Refinement datasets. It was observed that RAP-Gen, with its vocabulary and semantic-based retriever, significantly improved performance. Specifically, RAP-Gen-base using "Hybrid" improved perfect match performance beyond the best-performing baseline in TFix (49.70 → 54.15), and improved perfect match performance in the Code Refinement benchmark's small set (24.04 → 24.80) and medium set (14.18 → 15.84). All of these results validate that Search Enhanced Patch Generation (RAP-Gen) is an effective approach for APR.

[0073] This experiment demonstrates that search-enhanced patch generation using CodeT5 is an effective approach for program remediation. First, CodeT5 is compared to traditional APR techniques on the TFix benchmark, improved with deduplication of data and more appropriate evaluation metrics. Next, two sizes of CodeT5 integrated with the RAP-Gen framework are evaluated on the TFix and Code Refinement benchmarks. Furthermore, the experiment shows that the patch retriever finds relevant patches in terms of lexical and semantic similarity. In addition, a case study is provided to demonstrate how the retrieved bug fix patterns are useful for program remediation. Moreover, as the experiment demonstrates, the RAP-Gen framework provides improved performance for various error types and fix patterns. A detailed performance breakdown for 52 error types is listed, and error types that do not benefit from search enhancement in RAP-Gen are examined. Finally, the model is studied to see how it performs with a trivial but dominant error line removal fix pattern that simply removes error lines from bug-containing code.

[0074] This experiment demonstrates that search-enhanced patch generation using CodeT5 is an effective approach for program repair. First, TFix evaluation is improved. The original TFix benchmark uses the direct average of exact match (EM) accuracy across 52 error types as its primary evaluation metric. However, as shown in Table 7 in Figure 14, these error types have a fairly unbalanced distribution; for example, there are 16,217 instances of the major error type "no-invalid-this" compared to only 10 instances of the minor error type "no-new-symbol". Therefore, in some embodiments, a weighted average is employed to take into account the error type distribution. Furthermore, an examination of the released code regarding how TFix calculates exact matches revealed another limitation: if the predicted fix contains even one more whitespace, such as a space or new line, than the ground truth fix, it is considered a false exact match. However, in the JavaScript language, extra whitespace does not affect the correctness of the program. Therefore, a better metric is proposed: a weighted average of EM w / o spaces, which normalizes whitespace before calculating EM to eliminate the impact of mismatches in multiple whitespace. The TFix dataset has a duplication issue, so results for its deduplicated version are also included. Apart from perfect match accuracy, a looser metric of the BLEU-4 score is used to measure the subsequent overlap between the predicted correction and its ground truth correction. Note that the BLEU-4 score is also calculated after blank normalization.

[0075] As shown in Table 2 of Figure 9, the CodeT5 models are compared to other training-based baselines in TFix. One key observation is that, for the original mean EM w / spaces metric, CodeT5-base (50.88) yields better accuracy than T5-large (49.33), despite T5-large having a much larger model size (approximately 3.5 times that of CodeT5-base). Furthermore, focusing on a reasonable direct mean EM w / o spaces, CodeT5-base significantly improves performance, improving absolute accuracy by about 5 over T5-large (49.35 → 54.30). Based on weighted mean EM w / o spaces, both CodeT5-small (50.31) and CodeT5-base (53.57) outperform all baselines, including T5-large (49.70). This indicates that CodeT5 models with pre-training for code recognition on a large code corpus have a better understanding of the programs. For TFix evaluation, EM is used to indicate the weighted average EM without spaces unless otherwise specified. For the BLEU-4 metric, it is well consistent with the exact match metric, and CodeT5-base also achieves state-of-the-art (SoTA) performance of 78.85 compared to the original TFix.

[0076] Next, we will discuss the ablation study observations. In the deduplication-free TFix dataset, performance consistently shows a slight decrease across various metrics. This is an expected phenomenon, as the overlap between the training and test splits in the original data (34 instances) leads to a data leakage issue, resulting in an inappropriate increase in performance. When error information, including error types and error messages, is removed, a consistent performance decrease is observed in both the CodeT5-small and CodeT5-base models, making it useful to inform the program repair model which types of errors need to be corrected.

[0077] Refer to Table 3 in Figure 10 for an evaluation of RAP-Gen against TFix. Table 3 shows the results of the RAP-Gen framework against a deduplication version of the TFix benchmark. First, a Random baseline is established by randomly searching for bug fix pairs from the codebase. The performance degradation of both RAP-Gen-small and RAP-Gen-base with random searching means that the randomly searched fix patterns cannot provide useful guidance signals for program repair. Next, RAP-Gen integrated with different retrievers is compared, including the use of the lexical-based retriever BM25, the semantic-based retriever DPR based on dense vector matching, and two ensemble methods combining them. As a result, all search augmentation approaches significantly improved performance for both exact matches and BLEU-4 for both the small and base models. This indicates that search augmentation generation is a viable and effective approach for APR, and that both semantic and lexical information are important for searching relevant fix patterns. In the case of ensemble methods, RAP-Gen-base using "Hybrid" yields the best improvement over T5-large (49.58 → 54.15 EM). This verifies that an ensemble approach that considers both lexical and semantic information can combine the best of both worlds. Another observation is that the performance gain with search augmentation is greater with RAP-Gen-small than with RAP-Gen-base, which means that the improvement tends to saturate as the model size increases. Both RAP-Gen-small and RAP-Gen-base use the RAP-Gen framework with different patch generator backbones, specifically CodeT5-base and CodeT5-small, which have different model sizes, respectively.

[0078] In some embodiments, multiple methods may exist to fix a bug. Therefore, a perfect match with one ground truth patch is too strict a metric to consider other forms of correct fixes. To address this, a looser evaluation is used with an error removal metric according to TFix. Under this metric, a fixed patch is considered correct as long as it resolves the errors in the source bug-containing patch and does not introduce new errors (detected by the static analyzer ESLint). When attempting to reproduce this metric across 10,465 test instances, there are two difficulties: (1) Applying ESLint requires the full file context for each code patch, but it was found that 95 code files were unavailable for retrieval. (2) In some data samples, applying ESLint with the released configuration (https: / / github.com / eth-sri / TFix) results in parser errors. As a result, a filtered subset of 6,793 instances was curated by excluding those unavailable code files and samples with parser errors, where it was also found that fixes generated from TFix tend to have more parser errors. Referring to Table 4 in Figure 11, a comparison of error removal is shown. The RAP-Gen-small model is observed to perform significantly better than the T5-large model in error removal, which means that the RAP-Gen model is better able to synthesize different forms of good correction. Furthermore, in RAP-Gen-small, a misalignment is observed between the error removal metric and the perfect match metric, where the perfect match accuracy is low but the error removal accuracy is high. Such a misalignment is also observed in TFix.

[0079] Refer to Table 5 in Figure 12, which shows a comparison of the Code Refinement results with the methods described above. All baseline results (including the CodeT5 model) are obtained directly from their original papers. In "Naive Copy," the BLEU-4 score is quite high, but the perfect match (EM) is zero, which is observed to indicate a large overlap between the bug-containing code and its fix, suggesting that perfect match should be adopted as the primary evaluation metric. Among the baselines, NSEdit is highly competitive, producing the best results in the small subset (24.04 EM), while CodeT5-base with multitask training produces the best results in the medium set (14.18 EM).

[0080] RAP-Gen model comparisons reveal that RAP-Gen using various retrievers consistently outperforms their CodeT5 counterparts. The best model establishes new state-of-the-art results in two subsets (24.80 EM for small and 15.84 EM for medium), outperforming NSEdit by approximately 2 absolute points, particularly in the more challenging medium set. This reaffirms that retrieved correction patterns provide useful signals to guide program repair. Among the various retrievers, DPR outperforms BM25 for both RAP-Gen-small and RAP-Gen-base, revealing that semantic information may play a more important role than semantic information for this benchmark. Furthermore, "Hybrid" outperforms both BM25 and DPR, suggesting that the hybrid ensemble method is a more robust retriever for balancing both semantic and semantic information for this benchmark.

[0081] In summary, we conduct a comprehensive experiment to compare RAP-Gen with traditional learning-based methods across two benchmarks. First, we evaluate the CodeT5 model against TFix, improving its evaluation by providing a deduplication version of the dataset and more rational metrics, as well as introducing looser metrics for exact matches and matched BLEU-4 scores. As a result, CodeT5-base establishes new state-of-the-art performance on this task, improving EM from 49.70 to 53.57 compared to T5-large, and BLEU-4 from 76.98 to 78.85. Next, we evaluate the RAP-Gen model on both the TFix dataset and the code refinement dataset, observing that RAP-Gen, with its vocabulary and semantic-based retrievers, significantly improves performance. Specifically, RAP-Gen-base using "Hybrid" improved perfect match performance beyond the best-performing baseline in TFix (49.70 → 54.15), and improved perfect match performance in the small set of the Code Refinement benchmark (24.04 → 24.80) as well as in the medium set (14.18 → 15.84). All of these results verify that search-enhanced patch generation (RAP-Gen) using CodeT5 is an effective approach for APR.

[0082] Next, an experiment is conducted to evaluate whether the patch retriever can find relevant correction patterns that are helpful in program repair. First, an automated assessment is provided to measure the relevance of lexical and semantic similarities between the query and the retrieved patch. Furthermore, a specific case is provided to understand how the retrieved correction patterns contribute to better APR.

[0083] Refer to Table 6 in Figure 13, which shows the evaluation of the retriever. The retriever is analyzed for lexical and semantic matching between the query and the top retrieved patches. For lexical matching, BLEU-4 scores are used to measure their subtoken overlap, and for semantic matching, the cosine similarity (CosSim) between those dense vectors encoded by the fine-tuned DPR retriever is used. Table 6 in Figure 13 shows the performance of the patch retriever on both TFix and Code Refinement benchmarks. The first row shows the lower bound performance by randomly searching for bug fix pairs from the codebase, and it is observed that this Random baseline achieves much lower scores in both lexical and semantic matching. For lexical matching, BM25 performs better than DPR on TFix, but underperforms on two Code Refinement subsets. This may be due to data differences between TFix and Code Refinement, the latter employing obfuscated identifiers (e.g., VAR1, VAR2, ...) that hinder the performance of the lexical-based BM25 retriever. The hybrid retriever achieves the best lexical matching across all datasets, demonstrating that semantic information can complement lexical matching.

[0084] In the case of semantic matching, DPR achieves the best results for all datasets, which is not surprising since they are optimized for the same purpose. In particular, the hybrid retriever achieves slightly lower results than DPR but far better results than BM25, which means that the hybrid retriever may be more robust than lexical-based retrievers that balance both lexical and semantic information and are sensitive to the choice of identifier naming.

[0085] Referring again to Figures 6 and 7, case studies are used to demonstrate how found remediation patterns are useful in program repair, including case studies on TFix (Figure 6) and Code Refinement (Figure 7), where the RAP-Gen model with search extensions predicts the correct remediation, while CodeT5 without search extensions cannot. As shown in Figure 6, the found bug remediation patterns are precisely what is needed to remediate the source bug-containing code. Without search extensions, CodeT5 would incorrectly remove ".classify()" from the buggy line, probably through learning from the preceding adjacent lines. In the case of Code Refinement in Figure 7, the found bug remediation pair provides enough information to guide the RAP-Gen model to remediate the source bug-containing code. Without search extensions, CodeT5 would perform the incorrect remediation by simply removing the last line of code.

[0086] Therefore, both quantitative (Table 6 in Figure 13) and qualitative (Figures 6 and 7) results are obtained to evaluate the performance of the patch retriever and the corresponding automated programmed repair system. The results showed that the hybrid patch retriever is more robust and capable of finding lexical and semantically relevant patches to assist the programmed repair system.

[0087] Refer to Table 7 in Figures 8 and 15 to describe the performance of RAP-Gen for various error types and correction patterns. First, regarding the breakdown of its performance for different error types, a detailed breakdown of program repair performance on a deduplication-free TFix dataset is listed in Table 7 in Figure 15. CodeT5-base outperforms the previous SoTA T5-large in 44 out of 52 error types. In particular, for the major error type "no-invalid-this", CodeT5-base improves its perfect match from 37.48 in T5-large to 43.57, which corresponds to repairing 98 more instances. While T5-large can repair at least 50% of bugs for 44% of the 52 error types, CodeT5-base significantly increases this percentage to 60%, and RAP-Gen-small further improves it to 63%. Overall, RAP-Gen-base correctly fixes 478 bug-containing programs, more than the T5-large, in a much smaller model size.

[0088] Furthermore, we analyze the effect of search augmentation in RAP-Gen compared to the CodeT5 model for various error types. As shown in Table 7 of Figure 14, it is observed that search augmentation techniques have different effects on various error types, and can even impair program repair performance for certain subsets of error types. Specifically, in the experiment, it downgrades APR performance for 10 error types in CodeT5-small and 18 error types in CodeT5-base. Based on the number of exact match corrections, the largest performance downgrade in RAP-Gen-small is observed for the error type "no-extra-semi" (497 → 490), and the largest performance downgrade in RAP-Gen-base is observed for the error type "no-console" (228 → 220).

[0089] To investigate why search extensions can sometimes hinder perfect match performance in RAP-Gen models, we provide a case study of the "no-console" error type in Figure 8. In this case, ground truth remediation involves directly removing the error line within the bug-containing patch, and the RAP-Gen model repairs it into a different form based on the searched remediation pattern, which is counted as a false prediction with respect to perfect match. This once again confirms the limitations of the perfect match metric for evaluating program repair systems.

[0090] Next, the TFix benchmark is used to analyze which remediation patterns are performed by the model. After manually inspecting the bug remediation pairs, it is observed that the majority of remediations consist of deletion operations compared to code insertion and replacement operations. Bug remediation behaviors consist of code insertion (12.5%), replacement (8.1%), deletion (47.9%), insertion and replacement (6.9%), insertion and deletion (8.2%), replacement and deletion (7.2%), and all three methods (9.2%). Previous studies have also reflected that deletion operations are one of the most common remediation patterns. Among deletion operations, one dominant bug remediation pattern is error line removal, which is simply removing error lines from bug-containing code (as shown in the example in Figure 8). This trivial remediation pattern accounts for approximately 23% in the deduplication TFix test set. To further analyze this pattern, we compare how different models perform using the error line removal pattern, and the results are shown in Table 8. Using search extensions, we observe that RAP-Gen-base achieves the lowest false positive count of 56 (corresponding to the highest accuracy of 97.09), compared to CodeT5-base's 67 and T5-large's 71. This indicates that RAP-Gen can learn a wider variety of bug-fixing patterns rather than relying excessively on trivial error line removal patterns. Furthermore, RAP-Gen-small achieves the best recall and F1 score, but at the cost of predicting more false positives.

[0091] In summary, the difficulty of program repair varies depending on the error type. The best RAP-Gen-base in the experiment can repair 456 more bug-containing programs than the best-performing baseline T5-large. Error analysis is performed to analyze why search extensions can sometimes downgrade performance, and a case study is provided to show that this may be due to limitations of the exact match metric. Furthermore, one high-frequency correction pattern of error line removal is investigated to show that when handling this pattern, RAP-Gen-base gives the best accuracy score, while RAP-Gen-small achieves the best reproduction and F1 score.

[0092] This description and accompanying drawings illustrating aspects, embodiments, implementations, or applications of the present invention should not be construed as limiting. Various mechanical, compositional, structural, electrical, and operational modifications can be made without departing from the spirit and scope of this specification and the claims. In some cases, well-known circuits, structures, or techniques are not illustrated or described in detail so as not to obscure the embodiments of this disclosure. Similar numbers in two or more figures represent the same or similar elements.

[0093] This description provides specific details illustrating several embodiments consistent with the present disclosure. Numerous specific details are provided to provide a complete understanding of the embodiments. However, it will be apparent to those skilled in the art that some embodiments can be implemented without some or all of these specific details. The specific embodiments disclosed herein are illustrative and not limiting. Those skilled in the art can implement other elements not specifically described herein but within the scope and spirit of the disclosure. In addition, to avoid unnecessary repetition, one or more features illustrated and described in relation to one embodiment may be incorporated into other embodiments unless otherwise specifically described, or if one or more features would render the embodiment non-functional.

[0094] While exemplary embodiments have been shown and described, a wide range of modifications, alterations, and substitutions are contemplated in the foregoing disclosure, and in some cases, some features of the embodiments may be adopted without corresponding use of other features. Those skilled in the art will recognize many variations, substitutions, and modifications. Accordingly, the scope of the present invention should be limited only by the following claims, which should be interpreted to broadly coincide with the scope of the embodiments disclosed herein.

Claims

1. A method for automatic program repair, The steps include receiving a patch retriever, which includes a retriever encoder pre-trained using a first training dataset, by one or more processors, The steps include: receiving a patch generator containing a sequence generator neural network model pre-trained using a second training dataset by one or more of the aforementioned processors; A step of training the patch retriever and the patch generator using one or more processors with a two-stage training process, The first stage of training is performed by training the patch retriever using a third training dataset along with access to ground truth bug fix pairs. The second stage of training is performed by training the patch generator using a fourth training dataset with the patch retriever trained in the first stage of training, and during the second stage of training, the patch retriever is not allowed to access the bug fix pairs of the ground truth, and The steps include receiving a first bug-containing code patch by one or more of the aforementioned processors, The steps include: generating a first representation of the first bug-containing code patch using the retriever encoder of the patch retriever with one or more of the aforementioned processors; The steps of using the patch retriever with one or more processors to search for a first bug fix code pair from a first plurality of bug fix code pairs based on the first representation, The steps include generating a first extended bug-containing code patch based on the first bug-containing code patch and the first bug-fix code pair using one or more of the aforementioned processors, The steps include: generating a corrected code patch based on the first extended bug-containing code patch via a patch generator using one or more of the aforementioned processors; A method that includes this.

2. The method according to claim 1, wherein the patch retriever is configured to perform a search based on at least one of lexical similarity and semantic similarity with respect to the first bug-containing code patch.

3. The method according to claim 2, wherein the patch retriever is configured to perform a search based on a combination of lexical and semantic similarities with the first bug-containing code patch.

4. The method according to claim 1, wherein the patch generator includes a Transformer-based neural network model for sequence generation.

5. The method according to claim 1, wherein the first bug fix code pair of the first extended bug-containing code patch is used as a guide fix pattern for the patch generator.

6. The method according to claim 1, wherein the first training dataset includes a bug-containing patch and a corresponding corrected patch.

7. A non-temporary machine-readable medium containing multiple machine-readable instructions, wherein, when the multiple machine-readable instructions are executed by one or more processors, the one or more processors, A step of receiving a patch retriever which includes a retriever encoder that is pre-trained using a first training dataset, The steps include receiving a patch generator containing a sequence generator neural network model that is pre-trained using a second training dataset, A step of training the patch retriever and the patch generator using a two-stage training process, The first stage of training is performed by training the patch retriever using a third training dataset along with access to ground truth bug fix pairs. The second stage of training is performed by training the patch generator using a fourth training dataset with the patch retriever trained in the first stage of training, and during the second stage of training, the patch retriever is not allowed to access the bug fix pairs of the ground truth, and The first step is to receive a bug-containing code patch, The steps include generating a first representation of the first bug-containing code patch using the retriever encoder of the patch retriever, The steps of using the patch retriever to search for a first bug fix code pair from a first plurality of bug fix code pairs based on the first representation, The steps include generating a first extended bug-containing code patch based on the first bug-containing code patch and the first bug-fix code pair, The steps include generating a corrected code patch based on the first extended bug-containing code patch via a patch generator, and A non-temporary machine-readable medium adapted to perform a method including the following.

8. The non-temporary machine-readable medium according to claim 7, wherein the patch retriever is configured to perform a search based on one or more of the lexical and semantic similarities with the first bug-containing code patch.

9. It is a system, Non-temporary memory and One or more hardware processors coupled to the non-temporary memory and configured to read instructions from the non-temporary memory and cause the system to execute a method, The method comprises, A step of receiving a patch retriever which includes a retriever encoder that is pre-trained using a first training dataset, The steps include receiving a patch generator containing a sequence generator neural network model that is pre-trained using a second training dataset, A step of training the patch retriever and the patch generator using a two-stage training process, The first stage of training is performed by training the patch retriever using a third training dataset along with access to ground truth bug fix pairs. The second stage of training is performed by training the patch generator using a fourth training dataset with the patch retriever trained in the first stage of training, and during the second stage of training, the patch retriever is not allowed to access the bug fix pairs of the ground truth, and The first step is to receive a bug-containing code patch, The steps include generating a first representation of the first bug-containing code patch using the retriever encoder of the patch retriever, The steps of using the patch retriever to search for a first bug fix code pair from a first plurality of bug fix code pairs based on the first representation, The steps include generating a first extended bug-containing code patch based on the first bug-containing code patch and the first bug-fix code pair, The steps include generating a corrected code patch based on the first extended bug-containing code patch via a patch generator, and A system that includes this.

10. The system according to claim 9, wherein the patch retriever is configured to perform a search based on at least one of lexical similarity and semantic similarity with the first bug-containing code patch.

11. The system according to claim 10, wherein the patch retriever is configured to perform a search based on a combination of lexical similarity and semantic similarity with respect to the first bug-containing code patch.

12. The system according to claim 9, wherein the patch generator includes a Transformer-based neural network model for sequence generation.

13. The system according to claim 9, wherein the first bug fix code pair of the first extended bug-containing code patch is used as a guide fix pattern for the patch generator.

Images