Multimodal procedure automatic repair method, system, device, and storage medium

Abstract

The application discloses a kind of multimodal procedure automatic repair method, system, equipment and storage medium, method includes: by existing tool, source code is converted into abstract syntax tree AST;Anonymity variable and coding node;Build program repair model, the program repair model;The program repair model is pre-trained, and the pre-training includes two training tasks, one is to cover language model MLM task, let program repair model focus on the information of AST node level;Two are node order judgment NOJ task;After obtaining the trained program repair model, the multimodal procedure to be repaired is repaired using the trained program repair model.The application adopts the technical scheme of fusing multimodal information, solves the technical problem that model cannot learn the semantic, structural information of code sufficiently, so as to achieve the technical effect of fully utilizing the auxiliary information of code and finally making model learn more abundant semantic, structural information.

Description

Technical Field

[0001] This invention belongs to the technical field of program repair, specifically relating to a multimodal program automatic repair method, system, device and storage medium. Background Technology

[0002] Automatic program repair refers to fixing syntax and logic errors in code. Existing program repair methods mainly fall into two categories: static analysis, where a predefined code repair template is used to directly repair the program; and deep learning, which uses code context, test suites, and other auxiliary information to allow the model to learn the syntax and semantics of the code and then use this information to repair the program. However, existing technologies using deep learning for program repair suffer from the following main problems:

[0003] 1. Deficiencies of auxiliary information: Existing code often uses Abstract Syntax Tree (AST), comments, and code around defective code as auxiliary information, but these are not enough to describe the semantic information of the entire code.

[0004] 2. OOV: Out of Vocabulary, meaning that identifiers that were not learned in the training set appeared in the validation set. Summary of the Invention

[0005] The main objective of this invention is to overcome the shortcomings and deficiencies of the prior art and provide a multimodal program automatic repair method, system, device and storage medium to repair the program to be repaired more comprehensively.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides a method for automatic repair of multimodal programs, comprising the following steps:

[0008] The source code is converted into an abstract syntax tree (AST) using existing tools. The entire AST is traversed, and only variable names, method names, and statement type information are retained for node information, and only the relationships between nodes are retained for inter-node information. A simplified AST is generated in this way.

[0009] The variables in the source code are renamed according to their order of appearance. Simultaneously, the variable names, method names, and parameter information in the source code are saved to a "variable bucket," and the simplified AST is modified. The entire simplified AST is traversed to obtain path information, and each path is encoded using node encoding and leaf node position encoding. A program repair model is constructed, comprising 12 transformer encoder layers and employing a 12-head attention head mechanism. Each transformer encoder layer consists of an encoder module and a decoder module. Data first enters the encoder layer for encoding, then passes through a multi-head attention mechanism layer, a normalization layer, a feedforward neural network, and another normalization layer before entering the decoder layer for decoding. Finally, it passes through another multi-head attention mechanism layer, a normalization layer, a feedforward neural network, and another normalization layer to obtain the final result. Residual networks are used to connect the layers.

[0010] The program repair model is pre-trained, which includes two training tasks: one is a masked language model (MLM) task to allow the program repair model to focus on AST node-level information; the other is a node order judgment (NOJ) task. The program repair model is pre-trained using multimodal information, which consists of fused annotation information, code context, error information, and debugging information, specifically:

[0011] Use the entire function body containing the erroneous line of code as the code context, and represent the entire function body using a simplified AST;

[0012] The error message returned by the compiler is divided into two parts: the first part is the location of the error, and the second part is the type of error.

[0013] The debugging information returned by the compiler is divided into two parts: the first part is the variable name, and the second part is the value of the variable.

[0014] Comment information, error information and debugging information are encoded using BEP, code context is represented using a simplified AST, and each type of data is separated by the [SEP] flag to form multimodal information;

[0015] The program's automatic repair model is further trained using multimodal information and test feedback. The multimodal information is input into the program's automatic repair model, and new error information and the number of test cases passed are output. This step is repeated until the repaired code passes all test cases.

[0016] After obtaining the trained program repair model, the trained program repair model is used to repair the multimodal program to be repaired.

[0017] As a preferred technical solution, when deleting redundant information in the AST, each path is from the root node of the AST to the leaf node of the AST, and also includes the location information of the path. The AST is divided into nodes, and these nodes are encoded.

[0018] As a preferred technical solution, the process of converting the source code into an Abstract Syntax Tree (AST) using existing tools specifically involves:

[0019] A program consists of multiple functions; each function is converted into an AST using existing AST generation tools.

[0020] For each function, traverse the entire AST. If the type of a node is an identifier, then save the value of that node into the local variable bucket and make it anonymous. The method of making anonymous variables is to assign the variable names according to the order in which the variable names appear. At the same time, the parameter information of the method and the correspondence between the anonymous variable and the original variable must also be saved.

[0021] After traversing the entire program, the local variable buckets are merged into the global method bucket.

[0022] As a preferred technical solution, the pre-training of the program repair model specifically includes:

[0023] The source code is preprocessed, and an AST is generated using existing tools. Redundant data is removed, and variables are renamed according to their order of appearance in the source code to generate a simplified AST. Additional information about the variables is saved: if the variable is a method, its method name is saved; otherwise, its variable name and value are saved.

[0024] The MLM (Multi-Level Model) task involves masking language models to allow programs to repair the semantic information of the code learned by the model. Nodes in a simplified AST are randomly masked using dynamic probabilities, and the dynamic probabilities are related to the position of the node in the AST.

[0025] The NOJ task for determining node order allows the program repair model to learn the structural information of the code. Specifically, two nodes A and B are randomly selected from a simplified AST, and the program repair model is trained to determine whether the node pair (A, B) is ordered.

[0026] As a preferred technical solution, dynamic probability is used to mask tokens in the masked language model (MLM) task. In the procedural repair model, each token calculates its attention value with all other tokens. The larger the attention value, the closer the relationship between the token and the target token. The calculation formula is as follows:

[0027]

[0028] q i The attention(t) represents the probability that the i-th token is covered, and n indicates that the code snippet contains n tokens. a ,t b ) represents the calculation of t a and t b The attention value, t a and t b This represents the a-th token and the b-th token;

[0029]

[0030] Represents the i-th path after the masking operation, REPLACE(p i ,q i The expression `[mask]` indicates that the `[mask]` flag is used according to probability `q`. i Perform a masking operation;

[0031]

[0032] A masked An abstract syntax tree (AST) representing the code snippet after the masking operation;

[0033] Use the following loss function:

[0034]

[0035] Where N represents the length of the code segment, x represents the token, and C represents all tokens in the entire source code. t This represents the t-th token.

[0036] As a preferred technical solution, the comment information refers to the comment data within the function body containing the erroneous line, represented using BPE encoding;

[0037] The code context refers to the entire function body containing the line of error, represented using a simplified AST.

[0038] The error message is generated by the compiler and includes the error location and error type, represented using BPE encoding;

[0039] The debugging information is generated by the compiler and includes variable names and values, represented using BPE encoding;

[0040] As a preferred technical solution, in the test feedback method, test cases are used to check whether the code is correct. If all test cases pass, the code is output; if not all test cases pass, the new error information, new debugging information, comment information, and code context output by the compiler are combined to form new multimodal information. The new multimodal information is used to continue training the model and optimize the model parameters.

[0041] Secondly, the present invention provides a multimodal program automatic repair system, including: an AST simplification module, an anonymous variable and node encoding module, a model building module, a model training module, and a program repair module;

[0042] The AST simplification module is used to convert source code into an abstract syntax tree (AST) using existing tools. It traverses the entire AST, retaining only variable names, method names, and statement type information for node information, and only the relationships between nodes for inter-node information, thus generating a simplified AST.

[0043] The anonymous variable and node encoding module is used to rename variables in the source code according to their order of appearance, and save the variable names, method names and parameter information in the source code to the "variable bucket". It also modifies the simplified AST. The entire simplified AST is traversed to obtain path information, and each path is encoded using node encoding and leaf node position encoding.

[0044] The model building module is used to build a program repair model, which includes 12 transformer encoder layers and adopts a 12-head attention mechanism. Each transformer encoder layer consists of an encoder module and a decoder module. Data first enters the encoder layer for encoding, then passes through a multi-head attention mechanism layer, a normalization layer, a feedforward neural network, and another normalization layer before entering the decoder layer for decoding. Finally, it passes through a multi-head attention mechanism layer, a normalization layer, a feedforward neural network, and another normalization layer to obtain the final result. Residual networks are used to connect the layers.

[0045] The model training module is used to pre-train the program repair model. The pre-training includes two training tasks: one is a masked language modeling (MLM) task, which allows the program repair model to focus on AST node-level information; the other is a node order judgment (NOJ) task. The program repair model is pre-trained using multimodal information, which consists of fused annotation information, code context, error information, and debugging information, specifically:

[0046] Use the entire function body containing the erroneous line of code as the code context, and represent the entire function body using a simplified AST;

[0047] The error message returned by the compiler is divided into two parts: the first part is the location of the error, and the second part is the type of error.

[0048] The debugging information returned by the compiler is divided into two parts: the first part is the variable name, and the second part is the value of the variable.

[0049] Comment information, error information and debugging information are encoded using BEP, code context is represented using a simplified AST, and each type of data is separated by the [SEP] flag to form multimodal information;

[0050] The program's automatic repair model is further trained using multimodal information and test feedback. The multimodal information is input into the program's automatic repair model, and new error information and the number of test cases passed are output. This step is repeated until the repaired code passes all test cases.

[0051] The program repair module is used to repair the multimodal program to be repaired after obtaining the trained program repair model.

[0052] Thirdly, the present invention provides an electronic device, the electronic device comprising:

[0053] At least one processor; and,

[0054] A memory communicatively connected to the at least one processor; wherein,

[0055] The memory stores computer program instructions that can be executed by the at least one processor, which enables the at least one processor to perform the multimodal program automatic repair method.

[0056] Fourthly, the present invention provides a computer-readable storage medium storing a program, which, when executed by a processor, implements the aforementioned multimodal program automatic repair method.

[0057] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0058] 1. This invention uses a variable bucket method to reduce the occurrence of OOV problems and narrow the search space of the final result. This invention adopts a variable bucket technical solution to solve the problem of excessively large search space of the final result in the prior art, thereby achieving the technical effects of reducing the probability of OOV problems, narrowing the search space of the final result, and accelerating the generation speed of the final result.

[0059] 2. This invention, from the perspective of programmers debugging, adopts a test feedback technical solution, which solves the technical problem that existing technologies cannot effectively use error information and debugging information. As a result, error information and debugging information can fully guide model training, and the model can generate code faster and better.

[0060] 3. This invention adopts a technical solution that integrates multimodal information, which solves the technical problem that the model cannot fully learn the semantic and structural information of the code, thereby achieving the technical effect of making full use of the auxiliary information of the code and ultimately enabling the model to learn richer semantic and structural information.

[0061] 4. This invention adopts a simplified AST technical solution, which solves the technical problem of redundant information in AST interfering with model training, thereby achieving the technical effect of accelerating model training and improving model performance. Attached Figure Description

[0062] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 This is a flowchart of the multimodal program automatic repair method according to an embodiment of the present invention;

[0064] Figure 2 This is a flowchart of code preprocessing in an embodiment of the present invention;

[0065] Figure 3 This is a flowchart illustrating the variable slot generation results in an embodiment of the present invention;

[0066] Figure 4 This is a structural diagram of the variable slot in an embodiment of the present invention;

[0067] Figure 5 This is a flowchart of the test feedback training process according to an embodiment of the present invention;

[0068] Figure 6 This is a structural diagram of multimodal information according to an embodiment of the present invention;

[0069] Figure 7 This is a flowchart of the multimodal program automatic repair system according to an embodiment of the present invention;

[0070] Figure 8 This is a schematic diagram of the structure of the multimodal automatic program repair system according to an embodiment of the present invention;

[0071] Figure 9 This is a structural diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0072] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort are within the scope of protection of the present application.

[0073] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0074] This application addresses the shortcomings of existing program repair models in terms of auxiliary information by considering almost all the auxiliary information in existing program repair models, as follows:

[0075] 1. Code context. The function body containing the error code; if the function body is too long, select the few or dozens of lines of code surrounding the error code.

[0076] 2. Control Flow Graph (CFG): The code contains many conditional statements and loop statements, and the logic of these statements affects the meaning expressed by the entire code body.

[0077] 3. Data Flow Graph (DFG): Data flow graphs can effectively represent the flow of data. Therefore, in functions with a lot of numerical computation, data flow graphs can greatly help the model learn code semantics.

[0078] 4. Abstract Syntax Tree (AST): An Abstract Syntax Tree can effectively express the semantics of code because its nodes and edges contain semantic information. Since an AST includes conditional statements and loop statements, it can be said that an Abstract Syntax Tree contains a control flow graph.

[0079] 5. Error message: This is the error message generated when the test case is run. It includes errors such as array out of bounds, null pointer, type conversion, missing parentheses, etc. It also indicates the location of the error. Although the error location is sometimes not accurate, the error message can guide the model to modify the code to a certain extent.

[0080] 6. Comments: In some logically complex function bodies, comments are crucial for programmers' understanding. Therefore, comments also help the model understand the code.

[0081] To address the shortcomings of existing program repair models, this application provides a multimodal program automatic repair method that uses existing AST generation tools to process source code. For each AST node, only key information is stored to simplify the AST. Given a source code, in the first step, the source code is converted into an AST using existing AST generation tools. In the second step, the entire AST is traversed, and each node stores only its type, child nodes, and value.

[0082] In the process of program repair, the Out-of-Vote (OOV) problem is an important issue in the field of natural language processing. Its research significance lies in avoiding the appearance of unknown characters in the test set that are not present in the training set. In actual programming, the probability of OOV problems increases significantly due to programmer-defined variable names, package names, etc. Therefore, anonymizing all variable and method names during dataset processing can greatly reduce the occurrence of OOV problems, but this also results in the loss of the semantic information contained in the variable names themselves. In addition to variable names, there are also fixed data such as numbers and strings. These data can be directly copied into the result statement. Although strings may contain some semantic information, the impact is very small, so they are not considered in this application.

[0083] like Figure 1 , Figure 7 As shown in the figure, this embodiment provides a multimodal program automatic repair method, which includes the following steps:

[0084] S1. Convert the source code into an abstract syntax tree (AST). Discard all data in the AST except for variable names, statement types, and relation types to generate a simplified AST. Traverse the entire simplified AST to obtain path information. Encode each path using node encoding and leaf node position encoding.

[0085] Furthermore, to mitigate the OOV problem, this embodiment uses variable bucketing and anonymous variables. Given a source code, such as... Figure 2 As shown, code preprocessing is performed, and the following operations are executed:

[0086] The first step is to use existing AST generation tools to convert the source code into an AST;

[0087] The second step is to traverse the entire AST. If the type of a node is an identifier, then the value of that node is saved in the local variable bucket and anonymized. The method for anonymizing variables is to assign names to variables according to their order of appearance, while also saving the parameter information of the method and the correspondence between the anonymous variables and the original variables.

[0088] The third step is to merge the local variable buckets into the global method bucket after traversing the entire dataset.

[0089] Understandably, the purpose of the variable bucket is to reduce the OOV problem while minimizing the impact of variable anonymization on the model. The benefits of doing so are:

[0090] 1. Anonymous variables can reduce the occurrence of OOV (Out of Variable) problems. Variable names are set by the programmer and are not specially processed by the compiler during the compilation stage. Therefore, there is a high degree of freedom in setting variable names, and anonymous variables can effectively reduce the occurrence of OOV problems.

[0091] 2. Narrowing the search space of results. When traversing all nodes, the variable bucket stores the correspondence between the original variables and anonymous variables, as well as the function names and their parameter information. The former is to facilitate the conversion of anonymous variables to original variables when the model generates the final code, enhancing code readability, because the code must ultimately conform to the programmer's coding habits; no one would name variables "var_1," "var_2," etc. The latter is to narrow the search space of results. Figure 3 As shown, perform the following operations:

[0092] The first step is to generate variable slots. The variable slot structure is as follows: Figure 4 As shown. The first slot is the method name, and the second slot is the parameter information.

[0093] The second step is to generate parameter information. The model will generate the number of parameters and the type of each parameter.

[0094] The third step is to determine if there are suitable candidates in the variable bucket. The variable bucket stores a lot of function parameter information, and the model can filter out many candidates based on the generated parameter information. Therefore, this algorithm can narrow down the search space of results, thereby improving search speed and accuracy. If no suitable result is found in the variable bucket, the algorithm automatically generates the final result.

[0095] To further reduce the occurrence of OOV problems, this embodiment uses a subword-level word segmenter, Byte Pair Encoding (BPE).

[0096] To address the shortcomings of code auxiliary information, this embodiment also considers error information. Although existing algorithms also use error information as auxiliary information, they differ from this method. This embodiment employs a "debugging feedback" approach, integrating the Data Flow Graph (DFG) and error messages to enable the model to learn code information more comprehensively. This embodiment takes a programmer's perspective, identifying and correcting errors through debugging after discovery, with the following steps:

[0097] Scenario: Correct code C, incorrect code D, test case t. Input the incorrect code D into the untested codes d1 and d2 obtained from the model, as follows:

[0098] 1. Run test case t using correct C code, and save the value of the variable at the time of each line of code.

[0099] 2. Run test case t using code d1, and save the values ​​of variables and compiler error messages for each line of code.

[0100] 3. Compare the runtime variable values ​​corresponding to the correct code C and code d1. If they differ, it indicates that there is likely a problem with that line of code and its surrounding code. If a compiler error message appears at this point, it also indicates that there is likely a problem with that line of code and its surrounding code.

[0101] 4. Inputting variable value pairs (including the correct value when code C runs correctly, and the incorrect value when code d1 runs incorrectly), error messages, code locations, and surrounding code into the model can guide the model to modify the code.

[0102] 5. The model generates new, untested code d2 based on auxiliary information.

[0103] 6. Repeat steps 1-5 until the model generates code that passes all test cases before ending the loop.

[0104] S2. Construct a program repair model, which includes 12 transformer encoder layers and employs a 12-head attention mechanism. Each transformer encoder layer consists of an encoder module and a decoder module. Data first enters the encoder layer for encoding, then passes through a multi-head attention mechanism layer, a normalization layer, a feedforward neural network, and another normalization layer before entering the decoder layer for decoding. Finally, after passing through the multi-head attention mechanism layer, a normalization layer, a feedforward neural network, and another normalization layer, the final result is obtained. Residual networks are used to connect the layers.

[0105] S3. Pre-train the program repair model. The pre-training includes two training tasks: one is the Mask Language Model (MLM) task, which allows the model to focus on AST node-level information; the other is the Node Order Judgment (NOJ) task.

[0106] Before pre-training the model, the data needs to be preprocessed. The preprocessing steps are as follows:

[0107] The source code is converted into an Abstract Syntax Tree (AST), and redundant information in the AST is removed to prevent the model from obtaining defective data. While simplifying the AST, anonymous variables are preserved. To ensure code readability, this embodiment stores these anonymous variables in local variable buckets. The local variable buckets contain variable names, method names, and parameter information. Using the generated simplified AST, the entire AST is traversed to save path information. Each path runs from the AST root node to a leaf node and also includes the path's position information. This embodiment divides the AST into individual nodes and encodes these nodes. The entire path is encoded using node encoding and leaf node position encoding; this is the innovative aspect of the program's model repair method in this embodiment.

[0108] The program repair model in this embodiment is an Encoder-Decoder architecture. Essentially, this embodiment takes a source sequence as input to obtain a target sequence, so code serialization is required. For a code snippet, [BG] is placed at the beginning of the sentence to indicate the start of a line of code, and [CLS] is placed at the end of the sentence.

[0109] C = [BG], t1, t2, ..., t n [CLS] (2-1)

[0110] Where n is the length of the code fragment, and t represents the token. During the pre-training phase, this embodiment uses an AST derived from the source code. After a simplification process, the node information in the AST is significantly reduced. As described in the introduction, this embodiment believes that the paths in the AST contain structural and semantic information from the programming language. Therefore, this embodiment uses paths to encode the AST.

[0111] A = p1, p2, ..., p m (2-2)

[0112] m is the number of paths. Each path consists of several distinct nodes.

[0113] p i =t1,t2,…,t ni

[0114] t1 is the root node of the path, and ni is the length of the i-th path. In this embodiment, the encodings of all nodes are combined to represent the path.

[0115] p i =Concat[t1,t2,…,t] ni ;i] (2-4)

[0116] In this embodiment, 'i' is used as the location information of the path. Here, 'i' represents the i-th leaf node from left to right in the Abstract Syntax Tree (AST) of the original code snippet, which is the i-th path.

[0117] Furthermore, this embodiment sets up two pre-training tasks. The first is the Mask Language Model (MLM) task, borrowed from BERT. However, since the AST is a tree structure, the MLM task actually masks node information. The purpose of the MLM task is to allow the model to focus on the node-level information of the AST. The second is the Node Order Judgment (NOJ) task. In this embodiment, two points (A, B) are randomly selected on the path of the AST. These two points can be randomly selected, so this node pair may be ordered or unordered. This embodiment asks the model to determine whether this node pair (A, B) is ordered, that is, to determine whether node A is the parent node of B. The NOJ task is to allow the model to learn the structural information of the AST.

[0118] In 2018, BERT, based on the Transformer, introduced a new positional encoding, leading to a breakthrough in the field of NLP. BERT includes two pre-training tasks, the first of which is Masked Language Model (MLM).

[0119] 1. With an 80% probability, replace the token with the [MASK] tag, for example, my dog ​​is hairy -> my dog ​​is [MASK]

[0120] 2. With a 10% probability, replace the token with a random word, for example, "my dog ​​is hairy" -> "my dog ​​is apple".

[0121] 3. With a 10% probability, keep the token unchanged, for example, change "my dog ​​is hairy" to "my dog ​​is hairy".

[0122] Even if a fixed probability is used to mask the token.

[0123] This embodiment uses dynamic probability to mask tokens. In the Transformer+attention model, each token calculates its attention value with all other tokens. The larger the attention value, the closer the relationship between the token and the target token.

[0124]

[0125] q i `attention(t)` represents the probability that the i-th token is covered, and `n` indicates that the code snippet contains n tokens. a ,t b ) represents the calculation of t a and t b The attention value.

[0126]

[0127] This represents the i-th path after the masking operation. REPLACE(p i ,q i The expression `[mask]` indicates that the `[mask]` flag is used according to probability `q`. i Perform a masking operation.

[0128]

[0129] A masked An abstract syntax tree (AST) representing a code segment after a masking operation.

[0130] Use the following loss function:

[0131]

[0132] Where N represents the length of the code snippet, and x represents the token.

[0133] S4, such as Figure 5 As shown, the pre-trained program repair model is trained using a "debugging feedback" method. This method integrates code context, error messages, and the Data Flow Graph (DFG), specifically:

[0134] S41. Use the entire function body as the context of the code and represent the entire function body using a simplified AST.

[0135] S42. Divide the error message returned by the compiler into two parts: the first part is the location of the error, and the second part is the type of error.

[0136] S43. Divide the debugging information returned by the compiler into two parts: the first part is the variable name, and the second part is the value of the variable.

[0137] S44. Use BEP encoding for comment information, error messages, and debugging information. Use a simplified AST representation for the code context, and separate each type of data with the [SEP] flag, such as... Figure 6 As shown.

[0138] Furthermore, this embodiment uses Transformer + Attention to build the model, along with a compiler and test cases. The compiler outputs error messages, and the test cases are used to check whether the corrected program output by the algorithm is correct. An "error program" refers to a program that fails one or more test cases. When the model receives an error program, it first uses the compiler to output the corresponding error messages, then inputs the error program along with the error messages into the model for training. Using the trained model, the erroneous program is input, and the model outputs the corrected program.

[0139] S5. After obtaining the trained program repair model, use the trained repair model to repair the multimodal program to be repaired.

[0140] It should be noted that, for the sake of simplicity, the aforementioned method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously.

[0141] Based on the same idea as the multimodal program automatic repair method in the above embodiments, the present invention also provides a multimodal program automatic repair system, which can be used to execute the above-described multimodal program automatic repair method. For ease of explanation, the structural schematic diagram of the multimodal program automatic repair system embodiment only shows the parts related to the embodiments of the present invention. Those skilled in the art will understand that the illustrated structure does not constitute a limitation on the device, and may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0142] Please see Figure 8 In another embodiment of this application, a multimodal program automatic repair system 100 is provided, which includes an AST simplification module 101, an anonymous variable and node encoding module 102, a model building module 103, a model training module 104, and a program repair module 105.

[0143] The AST simplification module is used to convert source code into an abstract syntax tree (AST) using existing tools. It traverses the entire AST, retaining only variable names, method names, and statement type information for node information, and only the relationships between nodes for inter-node information. In this way, a simplified AST is generated.

[0144] The anonymous variable and node encoding module is used to rename the variables in the source code according to their order of appearance, and save the variable names, method names and parameter information in the source code to the "variable bucket". It also modifies the simplified AST. The entire simplified AST is traversed to obtain path information, and each path is encoded using node encoding and leaf node position encoding.

[0145] The model building module is used to build a program repair model, which includes 12 transformer encoder layers and adopts a 12-head attention mechanism. Each transformer encoder layer consists of an encoder module and a decoder module. Data first enters the encoder layer for encoding, then passes through a multi-head attention mechanism layer, a normalization layer, a feedforward neural network, and another normalization layer before entering the decoder layer for decoding. Finally, it passes through a multi-head attention mechanism layer, a normalization layer, a feedforward neural network, and another normalization layer to obtain the final result. Residual networks are used to connect the layers.

[0146] The model training module is used to pre-train the program repair model. The pre-training includes two training tasks: one is a masked language modeling (MLM) task, which allows the program repair model to focus on AST node-level information; the other is a node order judgment (NOJ) task. The program repair model is pre-trained using multimodal information, which consists of fused annotation information, code context, error information, and debugging information. Specifically:

[0147] Use the entire function body containing the erroneous line of code as the code context, and represent the entire function body using a simplified AST;

[0148] The error message returned by the compiler is divided into two parts: the first part is the location of the error, and the second part is the type of error.

[0149] The debugging information returned by the compiler is divided into two parts: the first part is the variable name, and the second part is the value of the variable.

[0150] Comment information, error information and debugging information are encoded using BEP, code context is represented using a simplified AST, and each type of data is separated by the [SEP] flag to form multimodal information;

[0151] The program's automatic repair model is further trained using multimodal information and test feedback. The multimodal information is input into the program's automatic repair model, and new error information and the number of test cases passed are output. This step is repeated until the repaired code passes all test cases.

[0152] The program repair module is used to repair the multimodal program to be repaired after obtaining the trained program repair model.

[0153] It should be noted that the multimodal program automatic repair system of the present invention corresponds one-to-one with the multimodal program automatic repair method of the present invention. The technical features and beneficial effects described in the embodiments of the multimodal program automatic repair method described above are applicable to the embodiments of multimodal program automatic repair. For details, please refer to the description in the embodiments of the method of the present invention, which will not be repeated here.

[0154] Furthermore, in the implementation of the multimodal program automatic repair system in the above embodiments, the logical division of each program module is only an example. In actual applications, the above functions can be assigned to different program modules as needed, for example, for the configuration requirements of the corresponding hardware or for the convenience of software implementation. That is, the internal structure of the multimodal program automatic repair system is divided into different program modules to complete all or part of the functions described above.

[0155] Please see Figure 9 In one embodiment, an electronic device is provided for implementing a multimodal program automatic repair method. The electronic device 200 may include a first processor 201, a first memory 202 and a bus, and may also include a computer program, such as a multimodal program automatic repair program 203, stored in the first memory 202 and executable on the first processor 201.

[0156] The first memory 202 includes at least one type of readable storage medium, including flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the first memory 202 can be an internal storage unit of the electronic device 200, such as the portable hard drive of the electronic device 200. In other embodiments, the first memory 202 can also be an external storage device of the electronic device 200, such as a plug-in portable hard drive, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device 200. Furthermore, the first memory 202 can include both internal storage units and external storage devices of the electronic device 200. The first memory 202 can be used not only to store application software and various types of data installed on the electronic device 200, such as the code of the multimodal program automatic repair program 203, but also to temporarily store data that has been output or will be output.

[0157] In some embodiments, the first processor 201 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The first processor 201 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the first memory 202 and calls data stored in the first memory 202 to perform various functions of the electronic device 200 and process data.

[0158] Figure 9 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 9 The structure shown does not constitute a limitation on the electronic device 200, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0159] The multimodal program auto-repair program 203 stored in the first memory 202 of the electronic device 200 is a combination of multiple instructions. When run in the first processor 201, it can achieve the following:

[0160] The source code is converted into an abstract syntax tree (AST) using existing tools. The entire AST is traversed, and only variable names, method names, and statement type information are retained for node information, and only the relationships between nodes are retained for inter-node information. A simplified AST is generated in this way.

[0161] The variables in the source code are renamed according to their order of appearance. Simultaneously, the variable names, method names, and parameter information in the source code are saved to a "variable bucket," and the simplified AST is modified. The entire simplified AST is traversed to obtain path information, and each path is encoded using node encoding and leaf node position encoding. A program repair model is constructed, comprising 12 transformer encoder layers and employing a 12-head attention head mechanism. Each transformer encoder layer consists of an encoder module and a decoder module. Data first enters the encoder layer for encoding, then passes through a multi-head attention mechanism layer, a normalization layer, a feedforward neural network, and another normalization layer before entering the decoder layer for decoding. Finally, it passes through another multi-head attention mechanism layer, a normalization layer, a feedforward neural network, and another normalization layer to obtain the final result. Residual networks are used to connect the layers.

[0162] The program repair model is pre-trained, which includes two training tasks: one is a masked language model (MLM) task to allow the program repair model to focus on AST node-level information; the other is a node order judgment (NOJ) task. The program repair model is pre-trained using multimodal information, which consists of fused annotation information, code context, error information, and debugging information, specifically:

[0163] Use the entire function body containing the erroneous line of code as the code context, and represent the entire function body using a simplified AST;

[0164] The error message returned by the compiler is divided into two parts: the first part is the location of the error, and the second part is the type of error.

[0165] The debugging information returned by the compiler is divided into two parts: the first part is the variable name, and the second part is the value of the variable.

[0166] Comment information, error information and debugging information are encoded using BEP, code context is represented using a simplified AST, and each type of data is separated by the [SEP] flag to form multimodal information;

[0167] The program's automatic repair model is further trained using multimodal information and test feedback. The multimodal information is input into the program's automatic repair model, and new error information and the number of test cases passed are output. This step is repeated until the repaired code passes all test cases.

[0168] After obtaining the trained program repair model, the trained program repair model is used to repair the multimodal program to be repaired.

[0169] Furthermore, if the modules / units integrated in the electronic device 200 are implemented as software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0170] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0171] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0172] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A multimodal program automatic repair method, characterized in that, Includes the following steps: The source code is converted into an abstract syntax tree (AST) using existing tools. The entire AST is traversed, and only variable names, method names, and statement type information are retained for node information, and only the relationships between nodes are retained for inter-node information. A simplified AST is generated in this way. The variables in the source code are renamed according to their order of appearance, and the variable names, method names, and parameter information in the source code are saved to a "variable bucket". The simplified AST is modified. The entire simplified AST is traversed to obtain path information, and each path is encoded using node encoding and leaf node position encoding. A program repair model is constructed, which includes 12 transformer encoder layers and adopts a 12-head attention head mechanism. Each transformer encoder layer consists of an encoder module and a decoder module. The data first enters the encoder layer for encoding, and after passing through a multi-head attention mechanism layer, a normalization layer, a feedforward neural network, and a normalization layer, it enters the decoder layer for decoding. After passing through a multi-head attention mechanism layer, a normalization layer, a feedforward neural network, and a normalization layer, the final result is obtained. The layers are connected by a residual network. The program repair model is pre-trained, which includes two training tasks: one is a masked language model (MLM) task to allow the program repair model to focus on AST node-level information; the other is a node order judgment (NOJ) task. The program repair model is pre-trained using multimodal information, which consists of fused annotation information, code context, error information, and debugging information, specifically: Use the entire function body containing the erroneous line of code as the code context, and represent the entire function body using a simplified AST; The error message returned by the compiler is divided into two parts: the first part is the location of the error, and the second part is the type of error. The debugging information returned by the compiler is divided into two parts: the first part is the variable name, and the second part is the value of the variable. Comment information, error information and debugging information are encoded using BEP, code context is represented using a simplified AST, and each type of data is separated by the [SEP] flag to form multimodal information; The program's automatic repair model is further trained using multimodal information and test feedback. The multimodal information is input into the program's automatic repair model, and new error information and the number of test cases passed are output. This step is repeated until the repaired code passes all test cases. After obtaining the trained program repair model, the trained program repair model is used to repair the multimodal program to be repaired.

2. The multimodal program automatic repair method according to claim 1, characterized in that, When removing redundant information from the AST, each path is from the root node of the AST to a leaf node of the AST, and also contains the location information of the path. The AST is divided into individual nodes, and these nodes are encoded.

3. The multimodal program automatic repair method according to claim 1, characterized in that, The process of converting the source code into an Abstract Syntax Tree (AST) using existing tools specifically involves: A program consists of multiple functions; each function is converted into an AST using existing AST generation tools. For each function, traverse the entire AST. If the type of a node is an identifier, then save the value of that node into the local variable bucket and make it anonymous. The method of making anonymous variables is to assign the variable names according to the order in which the variable names appear. At the same time, the parameter information of the method and the correspondence between the anonymous variable and the original variable must also be saved. After traversing the entire program, the local variable buckets are merged into the global method bucket.

4. The multimodal program automatic repair method according to claim 1, characterized in that, The pre-training of the program repair model specifically involves: Preprocess the source code, generate an AST using existing tools, remove redundant data, rename variables according to their order of appearance in the source code, generate a simplified AST, and save additional information about the variables. If the variable is a method, save its method name; if the variable is of another type, save its variable name and value. The MLM (Multi-Level Model) task involves masking language models to allow programs to repair the semantic information of the code learned by the model. Nodes in a simplified AST are randomly masked using dynamic probabilities, and the dynamic probabilities are related to the position of the node in the AST. The NOJ task for determining node order allows the program repair model to learn the structural information of the code. Specifically, two nodes A and B are randomly selected from a simplified AST, and the program repair model is trained to determine whether the node pair (A, B) is ordered.

5. The multimodal program automatic repair method according to claim 1, characterized in that, In the MLM (Masked Language Modeling) task, dynamic probabilities are used to mask tokens. In the procedural repair model, each token calculates its attention value with all other tokens. The larger the attention value, the closer the relationship between the token and the target token. The calculation formula is as follows: This represents the probability that the i-th token is covered, where n indicates that the code snippet contains n tokens. Representative calculation and The attention value, and Indicates the first The token and the One token; This represents the i-th path after the masking operation. This indicates that the [mask] flag is used according to probability. Perform a masking operation; An abstract syntax tree (AST) representing the code snippet after the masking operation; Use the following loss function: Where N represents the length of the code snippet, x represents the token, and C represents all tokens in the entire source code. This represents the t-th token.

6. The multimodal program automatic repair method according to claim 1, characterized in that, The comment information refers to the comment data within the function body containing the erroneous line, represented using BPE encoding; The code context refers to the entire function body containing the line of error, represented using a simplified AST. The error message is generated by the compiler and includes the error location and error type, represented using BPE encoding; The debugging information is generated by the compiler and includes variable names and values, represented using BPE encoding.

7. The multimodal program automatic repair method according to claim 1, characterized in that, In the test feedback method, test cases are used to check if the code is correct, and if all test cases pass, the code is output. If not all tests pass, the new error messages, new debugging information, comment information, and code context output by the compiler are combined to form new multimodal information. The new multimodal information is then used to continue training the model and optimize the model parameters.

8. A multimodal automatic program repair system, characterized in that, include: The module includes an AST simplification module, an anonymous variable and node encoding module, a model building module, a model training module, and a program repair module. The AST simplification module is used to convert source code into an abstract syntax tree (AST) using existing tools. It traverses the entire AST, retaining only variable names, method names, and statement type information for node information, and only the relationships between nodes for inter-node information, thus generating a simplified AST. The anonymous variable and node encoding module is used to rename variables in the source code according to their order of appearance, and save the variable names, method names and parameter information in the source code to the "variable bucket". It also modifies the simplified AST. The entire simplified AST is traversed to obtain path information, and each path is encoded using node encoding and leaf node position encoding. The model building module is used to build a program repair model, which includes 12 transformer encoder layers and adopts a 12-head attention mechanism. Each transformer encoder layer consists of an encoder module and a decoder module. Data first enters the encoder layer for encoding, then passes through a multi-head attention mechanism layer, a normalization layer, a feedforward neural network, and another normalization layer before entering the decoder layer for decoding. Finally, it passes through a multi-head attention mechanism layer, a normalization layer, a feedforward neural network, and another normalization layer to obtain the final result. Residual networks are used to connect the layers. The model training module is used to pre-train the program repair model. The pre-training includes two training tasks: one is a masked language modeling (MLM) task, which allows the program repair model to focus on AST node-level information; the other is a node order judgment (NOJ) task. The program repair model is pre-trained using multimodal information, which consists of fused annotation information, code context, error information, and debugging information, specifically: Use the entire function body containing the erroneous line of code as the code context, and represent the entire function body using a simplified AST; The error message returned by the compiler is divided into two parts: the first part is the location of the error, and the second part is the type of error. The debugging information returned by the compiler is divided into two parts: the first part is the variable name, and the second part is the value of the variable. Comment information, error information and debugging information are encoded using BEP, code context is represented using a simplified AST, and each type of data is separated by the [SEP] flag to form multimodal information; The program's automatic repair model is further trained using multimodal information and test feedback. The multimodal information is input into the program's automatic repair model, and new error information and the number of test cases passed are output. This step is repeated until the repaired code passes all test cases. The program repair module is used to repair the multimodal program to be repaired after obtaining the trained program repair model.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores computer program instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the multimodal program automatic repair method as described in any one of claims 1-7.

10. A computer-readable storage medium storing a program, characterized in that, When the program is executed by the processor, it implements the multimodal program automatic repair method according to any one of claims 1-7.

Images