Software error localization method based on error propagation modeling and defect feature enhancement

Abstract

The present invention relates to a software error location method based on error propagation modeling and defect feature enhancement. The present invention first eliminates the interference of accidentally correct test cases on software error location to enhance defect features and obtain an optimized program spectrum. Then, a complex network model based on statements and test cases is constructed according to the program spectrum and the program execution context. The importance of nodes is calculated by a ranking algorithm. Different weights are then assigned to test cases based on the importance of nodes. Finally, the suspicion of statements is calculated by combining the test case weights and a coverage-based error location calculation formula. The present invention improves the efficiency of software error location by reducing the coverage of truly erroneous statements by accidentally correct test cases, and uses a complex network ranking algorithm to calculate the contribution of test cases to error location, thereby reducing the impact of multiple statements with the same error suspicion value on the error location efficiency and saving software development costs.

Description

Technical Field

[0001] The present invention belongs to the technical field of software testing, relates to software error location, and specifically relates to a software error location method based on error propagation modeling and defect feature enhancement. Background Art

[0002] Early software error location mostly used manual analysis methods such as setting breakpoints. Manual error location is not only difficult but also extremely time-consuming. Automated software error location can help developers save error location costs. Depending on whether test cases need to be executed, it can be divided into error location based on static analysis and error location based on dynamic analysis. Static analysis-based methods do not run the program under test, but only statically analyze the internal program structure, interface, or documentation of the program under test to determine the possible locations of erroneous statements in the program under test. Software error location based on dynamic analysis, on the other hand, analyzes the runtime state generated during program execution by executing test cases, determines the defect location based on changes in observed results during program execution, and generates a defect report, providing a reference for developers to quickly and effectively locate software errors.

[0003] Among dynamic error location methods, error location based on program spectrum has proven to be a low-cost and effective method. This is because it is independent of the program model and easy to implement. The program spectrum represents the coverage information of the program during runtime, reflecting the characteristic information of a certain code profile of the program. There is a certain relationship between the program spectrum and program behavior. By comparing the difference in code coverage between failed test cases and successful test cases, it can help locate software errors. For any program entity s i The coverage characteristics under the test case can be expressed as a four-tuple, namely N(s i )=(N cf ,N uf ,N cp ,N up ), where these four elements represent execution failure and overwriting s i The number of test cases that failed to execute and were not covered i The number of test cases that are successfully executed and cover s i The number of test cases that are successfully executed and not covered i The number of test cases. For a single statement in a program, the more times it is executed by failed test cases and the fewer times it is executed by passing test cases, the greater the possibility that the statement contains an error. This feature is used to perform statistical analysis on program statements to find program statements containing errors. The present invention fully considers the different importance of each test case, eliminates the assumption of traditional algorithms that each test case is equally important, and dynamically adjusts the algorithm according to the importance of the test case. The positioning effect obtained by this algorithm on different test sets is stable and accurate.

[0004] Like software development, test case design is a human intellectual endeavor. People can make mistakes or be inconsiderate when designing test cases, inevitably resulting in poor test case quality. Accidental correctness in test cases is a major factor affecting the effectiveness of code coverage-based fault location techniques. Accidental correctness refers to the situation where a line of code containing an error in a program is executed but produces no incorrect results. Existing methods for handling accidentally correct test cases increase the suspicion of correct statements while also increasing the suspicion of truly erroneous statements, resulting in less than ideal fault location results. Summary of the Invention

[0005] In response to the shortcomings of the above-mentioned spectrum-based error localization technology, the present invention proposes a software error localization method based on error propagation modeling and defect feature enhancement. A complex network is used to construct an error propagation model and calculate the test case weights. The test case weighting method is not limited to the test case coverage vector itself. The contribution of the test case to error localization is calculated by analyzing the error propagation between the program spectrum and program statements. In view of the negative impact of accidentally correct test cases on error localization, two accidentally correct test case processing methods are proposed to enhance defect features, obtain an optimized program spectrum, and thus improve the efficiency of software error localization.

[0006] The method of the present invention specifically comprises the following steps:

[0007] Step 1: Instrument the source program, execute test cases, and collect the program spectrum of the test cases, including execution results and execution coverage information;

[0008] Step 2: Reduce the negative impact of accidental correct test cases on error localization;

[0009] Step 3: Use the execution coverage information of successful and failed test cases to establish two directed unweighted complex network models G respectively P and G F ;

[0010] 3-1. Establish connections between test cases and statement nodes based on the program spectrum. If the test case t i Covering statements j Then node t i With s j There are bidirectional edges between them;

[0011] 3-2. Establish connections between statements and statement nodes based on the test case execution context, and connect statement nodes in the test case execution trace in a one-way reverse order;

[0012] Step 4: Calculate the complex network model G separately through the TrustRank ranking algorithmP and G F The importance of each node in the test case is calculated, and weights are assigned to successful and failed test cases based on the node importance, and the weighted statement execution quadruple information is calculated;

[0013] 4-1. Construct a transfer matrix to represent the random walk jump probability between nodes in the network model. P and G F The method of constructing the transfer matrix is the same;

[0014] For G F , construct a size N s ×N t The matrix S2T, where N s The number of statements representing the union of failed test case execution vectors, N t Indicates the number of failed test cases; if statement s i Cover t j , t j implement Statements, Indicates t j The number of statements executed, allocated Because t j implement Each statement is for t j The TR contribution value is Construct a N-sized t ×N s The matrix T2S, if t j Execute i , s i cover failed test cases, assigned Because s i implement When there are s failed test cases, each test case has s i The TR contribution value is Construct an N that represents the error propagation between statements s ×N s The transfer matrix S2S, when the statement s i With s j When there is an edge between ij =1 / N s , and normalize S2S by column;

[0015] 4-2. Construct a trust propagation vector, select test case nodes as seed nodes, and add additional information provided by the test case to control the node's wandering behavior in the network model. This allows the complex network to have a higher bias towards this set of seed nodes during random walks.

[0016] The form of the trust propagation vector is set as v s Represents the trust propagation vector of the statement, v t represents the trust propagation vector of the test case; since only the test case node is selected as the seed node, In G F In the test case, the fewer statements executed, the greater the weight should be, so in: c i Indicates t i The number of statements executed, since the successful test case does not meet the above analysis, P Assign the same weight value to successful test cases. where N P is the total number of successful test cases;

[0017] 4-3. Calculate the statement execution quadruple information. During each iteration, the statement node absorbs the importance of the connected test case node and the statement node pointing to it. The test case node modifies its own importance based on the importance of the connected statement node. The importance of the statement and test case nodes interact with each other until the importance of the entire network node is stable. For example, in the kth iteration, the calculation of the corresponding node importance is:

[0018]

[0019]

[0020] where w s and w t They represent the node importance vectors of statements and test cases respectively, and α is the damping coefficient, which is used to ensure that the node value transfer can be stable and continuous during the calculation iteration without interruption or infinite amplification. Run the above algorithm twice to get G F and G P The importance vector of the midpoint node is normalized after the iteration is completed:

[0021]

[0022] 4-4. Calculate the statement's four-tuple information vector using the modified test case weight information:

[0023]

[0024]

[0025] in and A vector representing the number of failed and successful test cases for each statement in the program, N f and Np Represents the total number of failed and successful test cases respectively;

[0026] Step 5: Calculate the suspicion level based on the sentence's quadruple information using the SBFL calculation formula, and sort the error location information in descending order of suspicion level. Sentences with high suspicion levels are considered more likely to be erroneous.

[0027] Preferably, the acquisition program spectrum described in step 1 is specifically as follows:

[0028] Use GCC to compile the source program in the Ubuntu system, dynamically execute test cases, and use the GCOV tool to collect the execution results and execution coverage information of the test cases to obtain the program spectrum.

[0029] Preferably, the step 2 of reducing the negative impact of accidental correct test cases on error localization is specifically as follows:

[0030] Perform intersection operation on the statement coverage vectors of all failed test cases, and define the resulting set of program statements as the fault base; define the accidentally correct test case as the successful test case that executes all statements in the fault base; use method 1 or method 2; where method 1 is: modify the position of the accidentally correct test case covering the statement in the fault base to uncovered; method 2 is: modify the position of the accidentally correct test case covering the statement outside the fault base to uncovered and re-mark the accidentally correct test case as a failed test case to obtain new program coverage information.

[0031] Preferably, the suspicion level is calculated according to the four-tuple information of the sentence using the SBFL calculation formula in step 5, as follows:

[0032] The suspicion degree is calculated based on the four-tuple information vector of the statement using the SBFL calculation formula, such as The error location information is sorted in descending order of suspicion, and statements with high suspicion are considered to be more likely to be erroneous.

[0033] Compared with the traditional method, the present invention has the following beneficial effects:

[0034] The present invention takes into account that the existence of accidental correct test cases will reduce the suspicion of real error statements, and thus reduces the negative impact of accidental correct test cases on error location by reducing the coverage of real error statements by accidental correct test cases.

[0035] The present invention considers that spectrum-based fault localization techniques implicitly assume that every test case is equally important. However, some infrequent coverage relationships (e.g., tests that cover a small number of program entities or program entities that cover a small number of tests) are more helpful for fault localization. Therefore, considering test case weights can more efficiently distinguish the contribution of test cases to fault localization, reducing the cost of fault localization.

[0036] This invention considers that spectrum-based error localization techniques only consider program coverage test case information, treating each program entity in isolation and ignoring the internal information and interactions between program entities. However, statements that interact with statements with higher suspicion levels should also be more suspicious. Therefore, considering error propagation can reduce the impact of multiple statements with the same error suspicion level on error localization efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 For example program code and program spectrum.

[0038] Figure 2 It is an overall schematic diagram of the present invention.

[0039] Figure 3 Correct test case flow chart for handling contingency.

[0040] Figure 4 Flowchart for calculating node importance of complex network model.

[0041] Figure 5 This is an experimental result diagram showing how the damping coefficient α in the present invention improves the error positioning performance.

[0042] Figure 6 This is a graph showing the experimental results of the present invention on improving the error localization performance based on the Tarantula formula. DETAILED DESCRIPTION

[0043] In order to test the positioning efficiency of the method proposed in this invention, an experimental verification is conducted using Siemens suit. The suit contains seven projects: print_tokens, print_tokens2, schedule, schedule2, replace, tcas, and tot_info. The defect locations, number of defects, and defect types in these programs to be tested are all known. Figure 1 The is_num_constant() of the print_tokens2 program in Siemens is used as an example program to explain the present invention in detail. Figure 2 The specific steps are as follows:

[0044] Step 1: Enter is_num_constant() and the test case set {t1, t2, t3, t4, t5, t6, t7}; ● represents the statement executed when running the test case, blank indicates no coverage, the last line is the execution result of the test case, and P and F represent whether the test case is a success or failure test case, respectively.

[0045] Step 2: Reduce the interference of accidental correct test cases. The flowchart is as follows Figure 3 As shown. The statement coverage spectra of all failing test cases are intersected, and the resulting set of program statements is defined as the fault base. Accidentally correct test cases are defined as successful test cases that execute all statements in the fault base. Method 1 modifies the positions of accidentally correct test cases that cover statements in the fault base to uncovered. Method 2 modifies the positions of accidentally correct test cases that cover statements outside the fault base to uncovered and relabels these accidentally correct test cases as failing test cases. In this example, the fault base FB = {s1, s2, s3}, and t3, t5, and t7 execute all statements in the fault base, representing accidentally correct test cases. For Method 1, the positions of t3, t5, and t7 executing the FB are modified to uncovered, reducing the number of successful test cases covering true error statements. For Method 2, t3, t5, and t7 are relabeled as failing test cases and their execution positions outside the FB are modified to uncovered, reducing the number of successful test cases covering true error statements while increasing the number of failing test cases covering true error statements.

[0046] Step 3: Taking method 1 in step 2 as an example, construct two directed unweighted complex network models G using the execution coverage information of successful and failed test cases respectively. P and G F .

[0047] 3-1. Establish connections between test cases and statement nodes based on the program spectrum. If t i Test case coverage j Statement node t i With s j There are bidirectional edges between them. For example, there are bidirectional edges between test case t1 and statement nodes s1, s2, s3, s4, s5, s6, and s8.

[0048] 3-2. Establish connections between statements and statement nodes based on the test case execution context. Analyze error propagation, consider the impact of program statement behavior anomalies and error propagation on global error suspicion, and establish the topological relationship between statement nodes in the network. The principle of error propagation between statements is: there are different degrees of error propagation between statements in the test case execution path and it is assumed that errors propagate from the bottom to the top of the program execution context. For example, the G F, test case t1, the execution path is {s1, s2, s3, s4, s5, s6, s8}, so there is an edge<s8,s6> ,<s6,s5> ,<s5,s4> ,<s4,s3> ,<s3,s2> ,<s2,s1> ; The execution path of test case t2 is {s1, s2, s3, s4, s5, s7}, adding edge<s7,s5> The execution path of test case t6 is {s1, s2, s3, s8}, and the edge<s8,s3> .

[0049] Step 4: Calculate the complex network model G separately through the TrustRank ranking algorithm P and G F The importance of each node in the test case is calculated, and weights are assigned to successful and failed test cases according to the node importance. The weighted statement execution quadruple information is calculated. The specific process is as follows: Figure 4 The specific calculation is as follows:

[0050] 4-1. First, construct a transfer matrix to represent the random walk jump probability between nodes in the network model. P and G F The method of constructing the transfer matrix is the same.

[0051] For G F , construct a matrix S2T of size 8×3, such as statement s1 covers t1, allocate S2T 11 =1 / 7. Because t1 executes 7 statements, each statement contributes 1 / 7 to t1's TR. Construct a 3×8 matrix T2S. If t1 executes s1, assign T2S 11 =1 / 3, because s1 executes 3 failed test cases, and each test case contributes 1 / 3 to the TR of s2; construct an 8×8 transfer matrix representing the error propagation between statements. When statement s i With s j When there is an edge between ij =1 / 8, and normalize S2S by column.

[0052]

[0053]

[0054]

[0055] Usually the number of test cases in a data set is much larger than the number of statements, so the error propagation matrix S25 between statements is generally a sparse matrix, and T2S and S2T have a greater impact on the importance of nodes in the network.

[0056] 4-2. Construct a trust propagation vector, select the test case node as the seed node, and add additional information provided by the test case to control the node's wandering behavior in the network model. This makes the complex network more biased towards this set of seed nodes during the random walk.

[0057] The form of the trust propagation vector is set as Because only test case nodes are selected as seed nodes,

[0058] In G F In the test case, the fewer statements executed, the greater the weight should be, so in:

[0059] Since the successful test case does not meet the above analysis, in G P Assign the same weight value to successful test cases. where N P is the total number of successful test cases.

[0060] The trust propagation vector for the example program should be set to

[0061] 4-3. Node importance calculation, because the TrustRank algorithm uses the seed node set to calculate the PageRank value, so the principle of setting the initial value of the node in the network is: the initial value of the seed node is set to the trust propagation vector, the initial value of other nodes is set to 0, and the damping coefficient α is set to 0.5. F and G P When the importance of the nodes in the network is calculated, in each iteration, the statement node absorbs the importance of the connected test case nodes and the statement nodes pointing to it. The test case node modifies its own importance based on the importance of the connected statement nodes. The importance of the statement and test case nodes interact with each other until the importance of the entire network stabilizes. For example, in the kth iteration, the calculation of the corresponding node importance is:

[0062]

[0063]

[0064] Run the above algorithm twice to get G F and G P The importance vector of the midpoint node is normalized after the iteration is completed:

[0065]

[0066] The statement node importance vector of the example program is

[0067] 4-4. Calculate the quadruple information of each statement using the modified test case weight information.

[0068]

[0069]

[0070] in and A vector representing the number of failed and successful test cases for each statement in the program, N f and N p Represents the total number of failed and successful test cases respectively. f =[1, 1, 0.99, 0.57, 0.57, 0.29, 0.26, 0.62, 0], e p =[1, 1, 0.66, 0.4, 0.4, 0.4, 0.18, 0.42, 0.3].

[0071] Step 5: Calculate the suspicion level based on the four-tuple information of each statement using the SBFL calculation formula, and sort the error location information in descending order according to the suspicion level. Taking the calculation formula as an example, the sentence suspicion order obtained by the present invention is {s3, s8, s7, s4, s5, s2, s1, s6, s9}, while the sentence suspicion order obtained by traditional Tarantula is {s4, s7, s8, s5, s3, s1, s2, s6, s9}. From the example, it can be seen that the suspicion ranking of the true error sentence s3 is improved by the present invention.

[0072] Figure 5 This figure shows the performance improvement of the WTFL method on 12 classic error locator formulas by adjusting α within the range of [0, 1.0]. The horizontal axis represents different values of α, and the vertical axis represents the percentage improvement in the minimum mean error weight (MWE) metric on different error locators.

[0073] Figure 6The distribution of the error localization absolute ranking results (Log2 logarithmic scale) of the 7 projects was further compared, and violin plots were drawn using the error localization formula Tarantula of the original SBFL and the WTccl and WTcc2 methods of the present invention. The horizontal axis represents different error localization methods, and the vertical axis represents the ranking level of the error statement, that is, the position of the error statement in the suspicion ranking list. The black legend in the violin plot reflects the rank distribution of the error statement, the middle line represents the median of the detected error absolute ranking position, and the upper and lower lines represent the upper and lower quartiles, respectively. It can be seen from the figure that compared with the original SBFL results, the median and quartiles of the present invention have moved downward, and the distribution of most error statements is also at a lower position, and the position of the violin plot is generally lower, indicating that the overall ranking of the error statement has improved. It can be seen that the present invention can effectively improve the positioning efficiency of the error localization method.

Claims

1. Software error localization method based on error propagation modeling and defect feature enhancement, Its characteristics include the following steps: Step 1: Instrument the source program, execute test cases, and collect the program spectrum of the test cases, including execution results and execution coverage information; Step 2: Reduce the negative impact of accidental correct test cases on error localization; Step 3: Use the execution coverage information of successful and failed test cases to establish two directed unweighted complex network models G respectively P and G F ; 3-1. Establish connections between test cases and statement nodes based on the program spectrum. If the test case t i Covering statements j Then t i With s j There are bidirectional edges between them; 3-2. Establish connections between statements and statement nodes based on the test case execution context, and connect statement nodes in the test case execution trace in a one-way reverse order; Step 4: Calculate the complex network model G separately through the TrustRank ranking algorithm P and G F The importance of each node in the test case is calculated, and weights are assigned to successful and failed test cases based on the node importance, and the weighted statement execution quadruple information is calculated; 4-1. Construct a transfer matrix to represent the random walk jump probability between nodes in the network model. P and G F The method of constructing the transfer matrix is the same; For G F , construct a size N s ×N t The matrix S2T, where N s Represents the union of the execution statement vectors of failed test cases, N t Indicates the number of failed test cases; if statement s i Cover t j , t j implement Statements, Indicates t j The number of statements executed, allocated Because t j implement Each statement is for t j The TR contribution value is ; Construct a N-sized t ×N s The matrix T2S, if t j Execute i , s i cover failed test cases, assigned Because s i implement When there are s failed test cases, each test case has s i The TR contribution value is Construct an N that represents the error propagation between statements s ×N s The transfer matrix S2S, when the statement s i With s j When there is an edge between ij =1 / N s , and normalize S2S by column; 4-2. Construct a trust propagation vector, select test case nodes as seed nodes, and add additional information provided by the test case to control the node's wandering behavior in the network model. This allows the complex network to have a higher bias towards this set of seed nodes during random walks. The form of the trust propagation vector is set as v s Represents the trust propagation vector of the statement, v t represents the trust propagation vector of the test case; since only the test case node is selected as the seed node, In G F In the test case, the fewer statements executed, the greater the weight should be, so in: c i Indicates t i The number of statements executed, in G P Assign the same weight value to successful test cases. where N P is the total number of successful test cases; 4-3. Calculate the statement execution quadruple information. During each iteration, the statement node absorbs the importance of the connected test case node and the statement node pointing to it. The test case node modifies its own importance based on the importance of the connected statement node. The importance of the statement and test case nodes interact with each other until the importance of the entire network node is stable. At the kth iteration, the calculation of the corresponding node importance is: where w s and w t They represent the node importance vectors of statements and test cases respectively, and α is the damping coefficient, which is used to ensure that the node value transfer can be stable and continuous during the calculation iteration without interruption or infinite amplification. Run the above algorithm twice to get G F and G P The importance vector of the midpoint node is normalized after the iteration is completed: 4-4. Calculate the statement's four-tuple information vector using the modified test case weight information: in and A vector representing the number of failed and successful test cases for each statement in the program, N f and N p Represents the total number of failed and successful test cases respectively; Step 5: Calculate the suspicion level based on the sentence's quadruple information using the SBFL calculation formula, and sort the error location information in descending order of suspicion level. Sentences with high suspicion levels are considered more likely to be erroneous.

2. The software error localization method based on error propagation modeling and defect feature enhancement according to claim 1, characterized in that: The program spectrum for collecting test cases described in step 1 is specifically: Use GCC to compile the source program in the Ubuntu system, dynamically execute test cases, and use the GCOV tool to collect the execution results and execution coverage information of the test cases to obtain the program spectrum.

3. The software error localization method based on error propagation modeling and defect feature enhancement according to claim 1, characterized in that: The negative impact of reducing accidental correct test cases on error localization described in step 2 is specifically as follows: Perform intersection operation on the statement coverage vectors of all failed test cases, and define the resulting set of program statements as the fault base; define the accidentally correct test case as the successful test case that executes all statements in the fault base; use method 1 or method 2; where method 1 is: modify the position of the accidentally correct test case covering the statement in the fault base to uncovered; method 2 is: modify the position of the accidentally correct test case covering the statement outside the fault base to uncovered and re-mark the accidentally correct test case as a failed test case to obtain new program coverage information.

4. The software error localization method based on error propagation modeling and defect feature enhancement according to claim 1, characterized in that: The suspicion level is calculated based on the four-tuple information of the statement using the SBFL calculation formula in step 5, as follows:

Images