A software defect repair method based on change derivation and identification restructuring

Abstract

The application discloses a software defect repairing method based on change derivation and identifier reconstruction. The application derives changes with higher frequency of use to generate more similar changes, and discards changes that are almost not used, so that the number of collected changes is greatly reduced in the change collection stage. Meanwhile, the similarity between changes and bug statements is measured to accelerate the change searching process and reduce the time for searching for a corresponding repair template. The application not only uses variables, fields, literal quantities and the like collected from source project files, but also splits identifiers with a camel case according to the camel case, and reconstructs the split sub-identifiers to obtain new identifiers. The application not only increases the number of selected identifiers when concretizing abstract changes, but also reduces the distance between a candidate patch and a correct patch to a certain extent.

Description

Technical Field

[0001] This invention addresses the issues of low accuracy and long repair time in template-based automatic program repair within the field of software quality and automation. It proposes a novel software defect repair method that aims to significantly reduce the number of changes that need to be collected for software defect repair and the number of incorrect candidate patches generated before a correct patch can be produced. Background Technology

[0002] In the software development process, software defects are almost inevitable. Fixing defective programs accounts for a significant portion of the software development cycle. Automating software defect fixing promises to greatly reduce development time and quickly resolve bugs.

[0003] In recent years, research in this area has yielded many results, mainly forming four categories of methods: heuristic search-based, semantic constraint-based, statistical analysis-based, and template-based repair methods. This paper focuses primarily on template-based repair methods. Template-based repair methods are highly targeted and do not generate obscure or difficult-to-understand patches. Therefore, they are a relatively practical method for software defect repair. However, current template-based repair methods require tens of thousands of repair patterns to extract repair templates during the preparation process. Of these extracted repair patterns, only a small portion are frequently used; most are rarely used or almost never used. More importantly, some repair templates are very similar to bug fixes but have slight differences, leading to their failure to be used. Consequently, they are treated as the same as other unrelated repair patterns. In fact, only a small part of the repair pattern needs to be changed, such as changing a field or modifying an operator, to achieve the correct repair template effect. Summary of the Invention

[0004] This invention proposes a software defect repair method based on change evolution and identifier reconstruction. By evolving frequently used changes, more similar changes are generated, while discarding rarely used changes. This significantly reduces the number of changes collected during the change collection phase. Furthermore, by measuring the similarity between changes and bug statements, the change retrieval process is accelerated, reducing the time spent finding corresponding repair templates. In software development, developers with good programming habits often follow certain development standards. In actual development, it is frequently necessary to define variables, fields, and literals. Common naming conventions include Pascal's notation, underscore notation, Hungarian notation, and camelCase, with camelCase being the most common. Each word separated by camelCase has a specific meaning. By splitting identifiers such as variables and fields, and simultaneously performing synonym substitution on the split literals, the probability of generating correct patches from abstract changes is increased. This is reflected in our change specification phase. When specifying abstract changes, we not only use variables, fields, literals, etc. collected from the source project files, but we also split the identifiers named in camelCase according to camelCase and reconstruct the sub-identifiers after splitting them to obtain new identifiers. This not only increases the number of identifiers to choose from when specifying abstract changes, but also reduces the distance between candidate patches and correct patches to a certain extent.

[0005] The method of the present invention specifically includes the following steps:

[0006] Step 1: Extract frequently occurring historical changes and their corresponding contexts from the open-source community by filtering based on the criteria of highest score, views, and stars.

[0007] By collecting frequently accessed changes and their corresponding contexts based on open-source community access frequency, we can avoid collecting change fix patterns that are rarely used.

[0008] Step 2: Identify the context of the collected changes and classify them according to their context, placing changes with the same context under the same change set;

[0009] Step 3: Evolve the changes in the change pool. The specific steps are as follows:

[0010] Step 3-1: Parse the changes into an abstract syntax tree

[0011] Step 3-2: Traverse all nodes in the abstract syntax tree and identify key nodes in the abstract syntax tree through the key node rule set;

[0012] The key node rule set is shown in the table below (partial):

[0013]

[0014] The above nodes were identified as key nodes.

[0015] Step 3-3: Evolve the changes to the identified key nodes according to the evolution rule set;

[0016] The set of derivation rules is shown in the table below (partial):

[0017]

[0018] Steps 3-4: Add the derived changes to the context of the original changes.

[0019] Establish attribute relationships between the derived changes from step 3-3 and the original changes, and add them to the context corresponding to the original changes.

[0020] The original node of the evolutionary change:

[0021] Node{id:x1,nodeType:type s ,content:ctnt s ,attr s (attr1,attr2,…,attr n )}

[0022] The new node * generated by the evolutionary change:

[0023] Node*{id:x t ,nodeType:type t ,content t :ctnt,attr t (attr1*,attr2*,…,attr n *)}

[0024] Step 4: Locate the defect and identify its context.

[0025] The Ochiai defect localization tool was used to locate defects, resulting in a list of suspicious statements ranked from highest to lowest score. Suspicious statements were then selected in order. The context of each suspicious statement was extracted and compared with the change context in the patch pool to find the best-matching context.

[0026] Step 4-1: Use the Ochiai defect location method to locate the defect. The statement suspicion score is calculated as follows:

[0027]

[0028] Where failed(s) represents the number of test cases that cover the failure of statement s, passed(s) represents the number of test cases that cover the success of statement s, and totalfailed represents the total number of failed test cases.

[0029] Step 4-2: Identify and locate the context of suspicious statements, and detect the corresponding change sequence set in the change pool according to the context.

[0030] Step 5: Sort the changed sequence set by measuring similarity.

[0031] Step 5-1: Calculate the similarity score between the original generalized bug statement and the changes in the change set based on the node type of the abstract syntax tree.

[0032]

[0033] Where k represents the total number of AST types in the original bug statement and the changes set, TNs(i) represents the node type of the original bug statement, and TNt(i) represents the node type changed in the changes set.

[0034] Step 5-2: Calculate the similarity between the original generalized bug statement and the longest common subsequence of changes in the change set.

[0035]

[0036] Where LSs is the original modified subsequence, and LST is the derived modified subsequence.

[0037] Step 5-3: Calculate the size to be repaired based on the modification size of the original change to the derived change. Since the original change is itself, it has the highest score.

[0038]

[0039] Where MFs is the original modified subsequence, and MFt is the derived modified subsequence.

[0040] Step 5-4: Normalize the similarity scores calculated in steps 5-1, 5-2, and 5-3, and then calculate the final similarity score.

[0041] Simi(Ns,Nt)=f(TNS,TNt)*f(LSs,LSt)*f(MFs,MFt)

[0042] Step 5-5: Sort the change sets according to the final similarity calculated in 5-4.

[0043] Rank_Change context ={change schange1, change2, ..., change n}, where change s Indicates the original change;

[0044] Step 6: Concretize the abstract changes to generate candidate patches.

[0045] Step 6-1: Obtain a local source code corpus lexicon by processing and parsing the source code files of the local project.

[0046] Step 6-2: Split and reconstruct the identifiers of the original bug statements (for example below) to obtain more similar identifiers.

[0047] Orders.insertFirstItem()

[0048] The original sequence is split into: {Orders, $insert$, $First$, $Item$}

[0049] Query the isomorphic word corpus:

[0050] {$insert$:$delete$,$replace$;$First$:$Second$,$Last$}

[0051] Using isomorphic words, we can obtain more recombinations of new identifiers.

[0052] {insertFirstItem:insertSecondItem,insertLastItem,deleteFirstItem,deleteSecondItem,deleteLastItem,replaceFirstItem,replaceSecondItem,replaceLastItem}

[0053] Step 6-3: Select candidate changes in order of their ranking scores, and concretize the abstract changes to generate candidate patches. The concretization description is as follows: Figure 5

[0054] Step 6-4: Select the identifier using the following four strategies.

[0055] 1. Measure the similarity between isomorphic words and identifiers; identifiers with higher similarity have higher priority for reuse.

[0056] 2. Identifiers used more frequently in the source project have higher priority.

[0057] 3. Combining the two strategies above, given the same frequency, identifiers with higher similarity are more likely to be reused.

[0058] Step 6-5: Generate candidate patches after abstracting and materializing changes.

[0059] Step 7: Verify candidate patches using cascading verification. The next level of verification will only proceed after the previous level has passed verification, in order to reduce patch verification time.

[0060] Preferably, the identification of the change context involves classifying changes in the change pool based on the context, specifically as follows:

[0061] Step 2-1: For the definition of context, we follow the existing method Kim et al.; we represent context by defining node type and node hash, where node type can represent an abstract context type, and node hash represents a more unique context class;

[0062] Step 2-2: Node Type Definition

[0063] Suppose r is the root node of the modified subtree, and the context of node a is represented by the node type type(r, a), where type(r, a) is defined as follows:

[0064]

[0065] Where a.loc is the syntax location of a;

[0066] Steps 2-3: Node Hash Definition

[0067] Let node a be an example:

[0068] hash(a)={a.type{h(c1),h(c2),…,h(c k )}}

[0069] Where c1, c2, ..., c k It is a child node of node n

[0070] Steps 2-3: Context representation is composed of node type and node hash.

[0071] Context CTX consists of three components C P C L C R Composition; let c be a changing root node, with each component C having a set of nodes S. S It is composed of concatenated node types t(n,c) of node n or node hash h(n); the composition of the context is as follows:

[0072] CTX = P:CP L:C L R:C R

[0073] Changes in the change pool are categorized according to their context, and changes with the same context are added to the same change sequence. Changes with the same context are also sorted by frequency to obtain a well-categorized change pool.

[0074] Preferably, the cascaded verification specifically refers to:

[0075] 1) First, compile the candidate patch; 2) If the compilation is successful, perform the trigger test; 3) If the trigger test is successful, run the relevant test; 4) If the relevant test is successful, run all the tests; Patches that pass all tests may still be incorrect due to overfitting issues and need to be manually checked. If a candidate patch is syntactically and semantically equivalent to a manual patch, then the patch is considered a correct patch.

[0076] The beneficial effects of this invention are:

[0077] 1. By modifying the template, the probability of successful application is increased, while the number of changes collected in the early stages is reduced.

[0078] 2. For a set of changes within the same context, the changes are ranked by calculating three types of similarity scores, so that changes with higher correctness are used earlier, thereby reducing the time required to find the optimal change.

[0079] 3. By considering the programming standards of the development process, the identifiers are split using the widely used camelCase naming convention. Using a homogeneous word corpus, sub-identifiers with similar semantics are replaced and reconstructed to form new identifiers. This expands the original identifiers and increases the probability of obtaining the correct patch by concretizing the abstract patch. Attached Figure Description

[0080] Figure 1 Overall flowchart of software defect repair method based on change evolution and identifier reconstruction;

[0081] Figure 2 Key node identification diagram;

[0082] Figure 3 Example diagram of the change and evolution;

[0083] Figure 4 Change the description of the sorting algorithm;

[0084] Figure 5 Modify the specific algorithm description. Detailed Implementation

[0085] The present invention will now be described in detail with reference to the accompanying drawings. The overall process of the present invention is shown in the accompanying drawings. Figure 1 As shown, the specific steps are as follows:

[0086] Step 1: Extract frequently occurring historical changes and their corresponding contexts from the open-source community by filtering based on the criteria of highest score, views, and stars.

[0087] Step 2: Generalize the extracted changes and build a well-classified change pool based on their contexts. Specifically: Step 2-1: For the definition of context, follow the existing method Kim et al.; represent context by defining node type and node hash, where node type can represent an abstract context type, and node hash represents a more unique context class;

[0088] Step 2-2: Node Type Definition

[0089] Suppose r is the root node of the modified subtree, and the context of node a is represented by the node type type(r, a), where type(r, a) is defined as follows:

[0090]

[0091] Where a.loc is the syntax location of a;

[0092] Steps 2-3: Node Hash Definition

[0093] Let node a be an example:

[0094] hash(a)={a.type{h(c1),h(c2),…,h(c k )}}

[0095] Where c1, c2, ..., c k It is a child node of node n

[0096] Steps 2-3: Context representation is composed of node type and node hash.

[0097] Context CTX consists of three components C P C L C R Composition; let c be a changing root node, with each component C having a set of nodes S. S It is composed of the node type t(n,c) of node n or the node hash h(n) concatenated;

[0098] The context is composed of the following forms:

[0099] CTX = P:C P L:CL R:C R

[0100] Changes in the change pool are categorized according to their context, and changes with the same context are added to the same change sequence. Changes with the same context are also sorted by frequency to obtain a well-categorized change pool.

[0101] Step 3: Parse the extracted changes into an abstract syntax tree (AST), and identify the key nodes of the AST based on the key node rule set, such as... Figure 2 As shown;

[0102] The key node rule set is shown in the table below.

[0103]

[0104] Step 4: Evolve the changes for key nodes according to the evolution rule set and add them to the corresponding upper and lower bounds of the change pool.

[0105] In the text, such as Figure 3 As shown; the set of derivation rules is represented as follows;

[0106]

[0107] The derived changes are linked to the attributes of the original changes and added to the changeset of the context corresponding to the original changes.

[0108] The original node of the evolutionary change:

[0109] Node{id:x1,nodeType:type s ,content:ctnt s ,attr s (attr1,attr2,…,attr n )}

[0110] The new node * generated by the evolutionary change:

[0111] Node{id:x t ,nodeType:type t ,content t :ctnt,attr t (attr1*,attr2*,…,attr n *)}

[0112] Step 5: First, use the Ochiai defect localization tool to obtain a set of suspected candidate repair statements ranked from highest to lowest score. Based on the context of the candidate repair statements, find the corresponding change sequence in the change pool. Perform three similarity measures on the change series within the same context, calculate the similarity of each derived change in the change set, and sort the changes according to similarity, so that changes more similar to the original bug statement are used earlier. Figure 4 As shown.

[0113] Step 6: The change sequence from Step 5 is concretized to generate specific candidate patches. One of three concretization strategies is used to generate candidate patches, such as... Figure 5 As shown; specifically:

[0114] Step 6-1: Obtain a local source code corpus lexicon by processing and parsing the source code files of the local project;

[0115] Step 6-2: Split and reconstruct the identifiers of the original bug statements according to camelCase to obtain more similar identifiers;

[0116] Step 6-3: Select candidate changes in order of their ranking scores and then specify them to generate candidate patches;

[0117] Specifically:

[0118] Step 6-3-1: Select identifiers using the following four strategies;

[0119] ①Measure the similarity between isomorphic words and identifiers; the higher the similarity, the higher the priority for reuse of the identifier.

[0120] ② Identifiers used more frequently in the source project have higher priority;

[0121] ③ Combining the two strategies above, when the frequency is the same, identifiers with higher similarity are more likely to be reused;

[0122] Step 6-3-2: Abstract the change and concretize it based on the identifier selected in step 7-3-1 to generate candidate patches;

[0123] Step 7: Compile the candidate patches, triggering tests, relevant tests, and all tests in that order. Only if the previous level of tests passes will the next level of tests proceed, reducing patch verification time. Repeat the verification process until a candidate patch passes all tests or the verification time limit is exceeded. Finally, check whether the plausible patch is syntactically and semantically equivalent to a manually written patch.

Claims

1. A software defect repair method based on change evolution and identifier reconstruction, characterized in that, The method specifically includes the following steps: Step 1: Extract frequently occurring historical changes and their corresponding contexts from the open-source community by filtering based on the criteria of highest score, views, and stars; Step 2: Identify the context of the change, and classify the changes in the change pool according to the context, placing changes with the same context under the same change set; Step 3: Parse the changes into an abstract syntax tree, and identify the key nodes in the syntax tree based on the key node rule set. The key node rule set is shown in the table below. Step 4; For the identified key nodes, the changes are derived according to the derivation rule set, as shown in the table below: The derived changes are linked to the attributes of the original changes and added to the changeset of the context corresponding to the original changes. The original node of the evolutionary change: Node { id: x1, nodeType : type s , content : ctnt s ,attr s (attr1, attr2, … ,attr n )} The new node * generated by the evolutionary change: Node { id: x t , nodeType : type t , content t : ctnt ,attr t (attr1*, attr2*,… ,attr n *)} Step 5: Locate the defect and identify its context. Compare the context of the suspicious statement with the change context in the patch pool to find the best matching context. Step 6: Sort the change sets by measuring similarity; Step 6-1: Calculate the similarity score between the original generalized bug statement and the changes in the change set based on the node type of the abstract syntax tree; Where k represents the total number of abstract syntax tree types of the original bug statement and the changeset. Indicates the node type of the original bug statement. Indicates the type of node changed in the change group; Step 6-2: Calculate the similarity between the original generalized bug statement and the longest common subsequence of changes in the change set; in It is the common subsequence of the original changes. Common subsequences of evolutionary changes; Step 6-3: Calculate the size to be repaired based on the size of the modification of the original change to the derived change. Since the original change is itself, it has the highest score. in It is a subsequence of the original change. It is a subsequence of evolutionary changes; Step 6-4: Normalize the similarity scores calculated in steps 6-1, 6-2, and 6-3, and then calculate the final similarity score, i.e., change the ranking score; Step 6-5: Rank the change sets based on the final similarity calculated in 6-4. _ Change context ={ change s ,change1, change2, … ,change n }; where change s Indicates the original change; Step 7: Concretize the abstract changes to generate candidate patches; Step 7-1: Obtain a local source code corpus lexicon by processing and parsing the source code files of the local project; Step 7-2: Reconstruct the identifiers of the original bug statements by splitting them according to camelCase to obtain more similar identifiers; Step 7-3: Select candidate changes in order of their ranking scores, and then specify them to generate candidate patches; specifically: Step 7-3-1: Select the identifier using the following three strategies; ①Measure the similarity between isomorphic words and identifiers; the higher the similarity, the higher the priority for reuse of the identifier. ② Identifiers used more frequently in the source project have higher priority; ③ Combining the two strategies above, when the frequency is the same, identifiers with higher similarity are more likely to be reused; Step 7-3-2: Abstract the change and concretize it based on the identifier selected in Step 7-3-1 to generate candidate patches; Step 8: Verify candidate patches.

2. The software defect repair method based on change evolution and identifier reconstruction according to claim 1, characterized in that: The aforementioned identification of the change context, and the classification of changes in the change pool based on the context, specifically includes: Step 2-1: For the definition of context, we follow the existing method Kim et al.; we represent context by defining node type and node hash, where node type represents an abstract context type and node hash represents a more unique context class; Step 2-2: Node type definition; Suppose r is the root node of the modified subtree, and the context of node a is represented by the node type type(r, a), where type(r, a) is defined as follows: Where a.loc is the syntax location of a; Steps 2-3: Node hash definition; Let node a be an example: Where c1, c2, ..., c k It is a child node of node n; Steps 2-4: The context representation is composed of node type and node hash; Context CTX consists of three components C P C L C R Composition; let c be a changing root node, with each component C having a set of nodes S. S It is composed of concatenated node types t(n,c) of node n or node hash h(n); the composition of the context is as follows: CTX = P:C P ,L:C L ,R:C R Changes in the change pool are categorized according to their context, and changes with the same context are added to the same change sequence. Changes with the same context are also sorted by frequency to obtain a well-categorized change pool.

3. The software defect repair method based on change evolution and identifier reconstruction according to claim 1, characterized in that: The Ochiai defect localization method is used to locate the defects.

4. The software defect repair method based on change evolution and identifier reconstruction according to claim 1, characterized in that: The specific method for locating defects is as follows: use the Ochiai defect location tool to locate defects, obtain a list of suspicious statements from highest to lowest suspicious score, and select suspicious statements in order.

5. The software defect repair method based on change evolution and identifier reconstruction according to claim 1, characterized in that: The verification process specifically includes: 1) First, compile the candidate patch; 2) If the compilation is successful, perform the trigger test; 3) If the trigger test is successful, run the relevant test; 4) If the relevant test is successful, run all the tests; Patches that pass all tests may still be incorrect due to overfitting issues and need to be manually checked. If a candidate patch is syntactically and semantically equivalent to a manual patch, then the candidate patch is considered a correct patch.

Images