Evaluation methods and related components for pairwise test case coverage redundancy

Abstract

This application discloses a method, apparatus, device, and medium for evaluating the coverage redundancy of pairwise combined test cases, relating to the field of software testing technology. The method includes: constructing an initial test case matrix based on an acquired test case set; re-encoding the initial test case matrix to obtain a target test case matrix, and constructing a parameter combination matrix, a coverage redundancy matrix, and a test case redundancy distribution matrix; updating the coverage redundancy matrix and the test case redundancy distribution matrix using the target test case matrix and the parameter combination matrix, and obtaining a coverage redundancy degree matrix based on the updated coverage redundancy matrix to characterize the proportion of pairwise combined coverage redundancy; displaying the updated test case redundancy distribution matrix and the coverage redundancy degree matrix, and evaluating the coverage redundancy of pairwise combined test cases in the test case set. This method can quickly and clearly display and evaluate the coverage capability of the test case set and the distribution of pairwise combined coverage redundancy.

Description

Technical Field

[0001] This invention relates to the field of software testing technology, and in particular to an evaluation method, apparatus, device, and medium for evaluating redundancy in pairwise combined test cases. Background Technology

[0002] Software testing is a comprehensive, all-encompassing test of a software system, performed on various combinations of parameters. Research shows that most program defects are caused by the values ​​of a particular pair of parameters. A pair of parameter combinations means that the combination formed by pairing parameters appears at least once in the test case set; pairwise parameter combination testing is an effective method for software testing.

[0003] Pairwise test cases are the core of system testing. Currently, test case generation tools and methods struggle to generate the simplest pairwise parameter combination test case sets for different system combinations. Algorithm or tool designers mostly optimize the optimization process or path based on other optimization algorithms, rarely studying the inherent characteristics of the test case set itself. These inherent characteristics of pairwise test cases can include their coverage and redundancy, i.e., whether the test case includes all possible pairwise combinations of different parameters and whether a certain parameter combination is repeated, causing redundancy.

[0004] In summary, how to determine and evaluate the coverage capability and coverage redundancy distribution of a test case set in pairwise combined testing, so as to provide a referenceable and traceable measure of test case redundancy based on the evaluation results for optimizing test cases, is a problem that needs to be solved. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a method, apparatus, device, and medium for evaluating the redundancy of test case coverage in pairwise combinations. This method can determine and evaluate the coverage capability and redundancy distribution of a test case set in pairwise combination testing, providing a referenceable and traceable metric for test case redundancy based on the evaluation results. The specific solution is as follows:

[0006] Firstly, this application discloses a method for evaluating the redundancy of test case coverage in pairwise combinations, including:

[0007] Obtain the test case set, and construct the corresponding initial test case matrix based on the test case set;

[0008] The initial test case matrix is ​​re-encoded according to preset rules to obtain the updated target test case matrix, and a parameter combination matrix is ​​constructed based on the number of parameters in the test case set to characterize the pairwise combination of parameters.

[0009] Based on the parameter value range in the target test case matrix, a coverage redundancy matrix is ​​constructed to characterize the number of times each pair of parameters in the test case set is covered, and a test case redundancy distribution matrix is ​​constructed to characterize the coverage redundancy distribution of each pair of parameters in each test case.

[0010] The coverage redundancy matrix and the test case redundancy distribution matrix are updated using the target test case matrix and the parameter combination matrix, and the elements of the updated coverage redundancy matrix are analyzed and statistically analyzed to obtain a coverage redundancy degree matrix that characterizes the coverage redundancy ratio of each pair of combinations.

[0011] The updated test case redundancy distribution matrix and coverage redundancy matrix are displayed, and the coverage redundancy of pairwise combinations of test cases in the test case set is evaluated.

[0012] Optionally, the step of re-encoding the initial test case matrix according to preset rules to obtain the updated target test case matrix includes:

[0013] Determine the number of rows in the initial test case matrix, and based on the number of rows, re-encode the elements of each column in the initial test case matrix in ascending order to obtain the updated target test case matrix.

[0014] Optionally, the process of re-encoding the elements in each column in ascending order based on the row number to obtain the updated target test case matrix further includes:

[0015] For each column of the initial test case matrix, determine whether there is a target element whose original value is consistent with the value of the element to be encoded.

[0016] If it exists, then the encoded value of the target element is used to assign a value to the current element to be encoded;

[0017] If it does not exist, then determine the target encoding value of the encoded element adjacent to the current element to be encoded in the current column, and assign a value to the current element to be encoded based on the target encoding value.

[0018] Optionally, the step of constructing a coverage redundancy matrix based on the parameter value range in the target test case matrix to characterize the number of times each pair of parameters in the test case set is covered includes:

[0019] The target number of rows and columns of the matrix to be constructed is determined based on the pairwise combination of parameters in the parameter combination matrix and the range of parameter values ​​in the target test case matrix.

[0020] Based on the target number of rows and columns, a coverage redundancy matrix is ​​constructed to characterize the number of times each pair of parameters in the test case set is covered.

[0021] Optionally, updating the coverage redundancy matrix and the test case redundancy distribution matrix using the target test case matrix and the parameter combination matrix includes:

[0022] Initialize the coverage redundancy matrix and the test case redundancy distribution matrix;

[0023] The initialized coverage redundancy matrix and test case redundancy distribution matrix are updated using the target test case matrix and the parameter combination matrix.

[0024] Optionally, updating the coverage redundancy matrix and the test case redundancy distribution matrix using the target test case matrix and the parameter combination matrix includes:

[0025] Based on each column of the parameter combination matrix, each row of the target test case matrix is ​​read sequentially to obtain the reading result;

[0026] The coverage redundancy matrix and the test case redundancy distribution matrix are updated based on the reading results.

[0027] Optionally, updating the coverage redundancy matrix and the test case redundancy distribution matrix based on the reading results includes:

[0028] Based on the reading result, the corresponding first target position is determined in the coverage redundancy matrix, and the current value of the first target position is increased by a preset unit value to obtain the updated value;

[0029] If the updated value is greater than the preset unit value, then the second target position in the test case redundancy distribution matrix is ​​determined, and the current value of the second target position is increased by the preset unit value.

[0030] Secondly, this application discloses an evaluation device for redundancy coverage of pairwise combined test cases, comprising:

[0031] The first matrix construction module is used to obtain a test case set and construct a corresponding initial test case matrix based on the test case set.

[0032] The second matrix construction module is used to re-encode the initial test case matrix according to preset rules to obtain the updated target test case matrix, and to construct a parameter combination matrix based on the number of parameters in the test case set to characterize the pairwise combination of parameters.

[0033] The third matrix construction module is used to construct a coverage redundancy matrix based on the parameter value range in the target test case matrix to characterize the number of times each pair of parameters in the test case set is covered, and to construct a test case redundancy distribution matrix to characterize the coverage redundancy distribution of each pair of parameters in each test case.

[0034] The matrix update statistics module is used to update the coverage redundancy matrix and the test case redundancy distribution matrix using the target test case matrix and the parameter combination matrix, and to analyze and statistically analyze the elements of the updated coverage redundancy matrix to obtain a coverage redundancy degree matrix that characterizes the coverage redundancy ratio of each pair of combinations.

[0035] The coverage redundancy evaluation module is used to display the updated test case redundancy distribution matrix and the coverage redundancy degree matrix, and to evaluate the coverage redundancy of pairwise combinations of test cases in the test case set.

[0036] Thirdly, this application discloses an electronic device, including:

[0037] Memory, used to store computer programs;

[0038] A processor is used to execute the computer program to implement the steps of the aforementioned evaluation method for covering redundancy in pairwise combined test cases.

[0039] Fourthly, this application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the steps of the aforementioned disclosed evaluation method for covering redundancy in pairwise combined test cases.

[0040] As can be seen, this application obtains a test case set and constructs a corresponding initial test case matrix based on the test case set; re-assigns and encodes the initial test case matrix according to preset rules to obtain an updated target test case matrix, and constructs a parameter combination matrix to characterize the pairwise combination of parameters based on the number of parameters in the test case set; constructs a coverage redundancy matrix to characterize the number of times each pairwise combination of parameters in the test case set is covered based on the parameter value range in the target test case matrix, and constructs a test case redundancy distribution matrix to characterize the pairwise combination coverage redundancy distribution in each test case; updates the coverage redundancy matrix and the test case redundancy distribution matrix using the target test case matrix and the parameter combination matrix, and analyzes and statistically analyzes the elements of the updated coverage redundancy matrix to obtain a coverage redundancy degree matrix to characterize the pairwise combination coverage redundancy ratio; displays the updated test case redundancy distribution matrix and the coverage redundancy degree matrix, and evaluates the coverage redundancy of the pairwise combination test cases in the test case set. Therefore, this application constructs an initial test case matrix based on the obtained test case set, and re-encodes the initial test case matrix to obtain the target test case matrix and constructs a parameter combination matrix to determine the pairwise combination of parameters; constructs a coverage redundancy matrix to determine the number of times each pairwise combination is covered, and constructs a test case redundancy distribution matrix to determine the coverage redundancy distribution of individual test cases in pairwise combinations; then, the coverage redundancy matrix and test case redundancy distribution matrix are updated using the target test case matrix and parameter combination matrix; then, the elements in the updated coverage redundancy matrix are analyzed and statistically analyzed to obtain a coverage redundancy degree matrix, which is used to determine the coverage redundancy ratio of each pairwise combination; finally, the updated test case redundancy distribution matrix and coverage redundancy degree matrix are displayed, and the coverage redundancy of each pairwise combination of test cases in the test case set is evaluated. In this way, the above scheme can quickly determine the coverage capability and coverage redundancy distribution of the test case set in pairwise combination testing, and provide a clear and intuitive display and evaluation, thereby providing a referenceable and traceable measure of test case redundancy for optimizing test cases based on the evaluation results. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0042] Figure 1 This application discloses a flowchart of a method for evaluating the redundancy of test case coverage through pairwise combinations.

[0043] Figure 2 This is a schematic diagram of a specific initial test case set matrix (TS) disclosed in this application;

[0044] Figure 3 This is a schematic diagram of a specific parameter combination matrix C disclosed in this application;

[0045] Figure 4 This application discloses a flowchart of a specific method for evaluating the redundancy of pairwise combined test cases.

[0046] Figure 5 This is a schematic diagram of the target test case matrix TS' after assignment and update as disclosed in this application;

[0047] Figure 6 This is a flowchart of an update algorithm for covering a redundancy matrix and a test case redundancy distribution matrix disclosed in this application;

[0048] Figure 7 This is a schematic diagram of a specific coverage redundancy matrix disclosed in this application;

[0049] Figure 8 This is a schematic diagram of a specific test case redundancy distribution matrix disclosed in this application;

[0050] Figure 9 This is a schematic diagram of a specific coverage redundancy matrix disclosed in this application;

[0051] Figure 10 This is a specific test case redundancy distribution curve disclosed in this application;

[0052] Figure 11 This application discloses a specific bar chart showing the coverage redundancy ratio.

[0053] Figure 12 This is a schematic diagram of an evaluation device for covering redundancy in pairwise combined test cases, as disclosed in this application.

[0054] Figure 13 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0055] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0056] Currently, test case generation tools and methods struggle to generate the simplest pairwise parameter combination test case sets for different system combinations. Algorithm or tool designers mostly optimize the optimization process or path based on other optimization algorithms, rarely studying the inherent characteristics of the test case set itself. Therefore, this application discloses a method, apparatus, device, and medium for evaluating the redundancy coverage of pairwise combined test cases. This method can determine and evaluate the coverage capability and redundancy distribution of a test case set in pairwise combined testing, providing a referenceable and traceable metric for test case redundancy based on the evaluation results.

[0057] See Figure 1 As shown in the embodiment of this application, a method for evaluating the redundancy of test case coverage in pairwise combinations is disclosed. The method includes:

[0058] Step S11: Obtain the test case set and construct the corresponding initial test case matrix based on the test case set.

[0059] In this embodiment, the test case set is first obtained. To more intuitively understand the characteristics of the test case set, the test cases are represented in matrix form to obtain the initial test case matrix corresponding to the test case set.

[0060]

[0061] This matrix contains m×n a's. i,j The element is defined as follows: m represents the number of test cases in the test case set, and n represents the number of parameters in the test case set. i,j This represents the value of the j-th parameter in the i-th test case. In other words, by sequentially reading each test case T in the test case set TS and describing the test case set TS into a matrix representation using mathematical modeling, the initial test case matrix can be obtained. For example, a specific test case TS is:

[0062] TS={T1,T2,T3,T4,T5,T6,T7,T8,T9};

[0063] Where, T1 = {0.8, -1.3, 1.7, 0}, T2 = {1.2, 0.6, 1.8, -5}, T3 = {2.5, 0.6, 9, 2},

[0064] T4={4,0,3,0.32}, T5={1.2,-1.3,1.8,2}, T6={0.8,0.6,1.7,0.32},

[0065] T7={4,0,3,0}, T8={2.5,3,1.8,-5}, T9={2.5,-1.3,9,0.32}.

[0066] Then the corresponding initial test case matrix is ​​as follows: Figure 2 As shown, the matrix consists of 9×4 numbers a. i,j The arranged number table, where a i,j This represents the value of the j-th parameter in the i-th test case, such as a. 3,1 =2.5 means that the first parameter of the third test case in this test case set has a value of 2.5.

[0067] Step S12: Re-encode the initial test case matrix according to the preset rules to obtain the updated target test case matrix, and construct a parameter combination matrix to represent the pairwise combination of parameters based on the number of parameters in the test case set.

[0068] In this embodiment, to facilitate subsequent calculations, the initial test case matrix needs to be re-encoded according to preset rules to obtain an updated target test case matrix. A parameter combination matrix is ​​then constructed based on the number of parameters in the test case set to characterize the pairwise combinations of parameters. This parameter combination matrix can also be understood as a parameter position matrix, used to characterize the pairwise combinations of parameters and to determine the two parameters to be combined and their specific values.

[0069] The size of the parameter combination matrix is ​​2×r, where, 'n' represents the number of parameters in the test case set. The first row of the matrix represents the position of the first parameter, the second row represents the position of the second parameter, and each column of the matrix represents a pairwise combination of parameters. For example, assuming there are 4 parameters in the test case set, the parameter combination matrix can be found in [reference needed]. Figure 3 As shown, Figure 3 The first column represents the combination of the first and second parameters, the second column represents the combination of the first and third parameters, and so on.

[0070] Step S13: Based on the parameter value range in the target test case matrix, construct a coverage redundancy matrix to characterize the number of times each pair of parameters in the test case set is covered, and construct a test case redundancy distribution matrix to characterize the coverage redundancy distribution of each pair of parameters in each test case.

[0071] In this embodiment, a coverage redundancy matrix (COV) is constructed based on the parameter value range in the target test case matrix to characterize the number of times each pair of parameters in the test case set is covered. The position of each element indicates the source of the combined parameters, and its value indicates the number of times the combination is covered. A test case redundancy distribution matrix R is also constructed to represent the pairwise coverage redundancy distribution in each test case. Its size is 2×m, where m represents the number of test cases in the test case set. Specifically, a 2×m test case redundancy distribution matrix is ​​constructed to store the coverage information of pairwise redundancy for individual test cases, as shown below:

[0072]

[0073] In this matrix, the first row ranges from 1 to m, representing the position of the currently read test case T in the test case set TS. The second row contains the values ​​for redundancy (COVER), where each number indicates how many redundant test cases are covered in pairwise combinations with all the preceding test cases.

[0074] Step S14: Update the coverage redundancy matrix and the test case redundancy distribution matrix using the target test case matrix and the parameter combination matrix, and analyze and statistically analyze the elements of the updated coverage redundancy matrix to obtain a coverage redundancy degree matrix that characterizes the coverage redundancy ratio of each pair of combinations.

[0075] In this embodiment, updating the coverage redundancy matrix and the test case redundancy distribution matrix using the target test case matrix and the parameter combination matrix includes: initializing the coverage redundancy matrix and the test case redundancy distribution matrix; and updating the initialized coverage redundancy matrix and the test case redundancy distribution matrix using the target test case matrix and the parameter combination matrix. That is, after constructing the coverage redundancy matrix and the test case redundancy distribution matrix, they need to be initialized, and then updated using the target test case matrix and the parameter combination matrix. After obtaining the updated coverage redundancy matrix, to more clearly express the proportion of redundancy in each pair of combinations, the elements in the updated coverage redundancy matrix are divided according to the number of times they are covered, and a matrix is ​​used to store the redundancy information, i.e., the coverage redundancy degree matrix U. Let e ​​be the number with the largest number of times it is covered in each pair of combinations + 1, i.e., e = max(COVER) + 1, and create a 3×e matrix to store the coverage information of the redundancy degree of each pair of combinations.

[0076]

[0077] The first row of this matrix represents the degree of redundancy in the pairwise combinations, such as u. 1,2=1 indicates a set where each pair of combinations in the test case set is covered once; the second line indicates the number of times each pair of combinations has redundant coverage, such as u 2,2 =0 indicates that the number of pairs of combinations covered once in this test case set is 0; the third line is used to indicate the proportion of this redundancy among all pairs of combinations, such as u 3,2 =0 indicates that the proportion of pairs of combinations that were covered once in the test case set is 0.

[0078] That is, by counting the number of different values ​​in the updated coverage redundancy matrix and recording them in the second row of U, the proportion of each value in the second row is calculated and recorded in the third row. Specifically, when calculating the value of the second row of U based on the redundancy of all elements in COV, the following steps are included: sequentially reading each row of the COV matrix, and sequentially reading each value c in each row of COV. i,j Compare the currently read c from left to right. i,j The corresponding value u in the first row of U 1,e until c i,j =u 1,e Then let u 2,e =u 2,e +1, meaning the current redundancy is u. 1,e Increment the number of pairwise combinations by one; repeat the above steps until all values ​​c in the current row have been processed. i,j Repeat all the above steps until all rows in COV have been processed. When calculating the value of the third row in U based on the redundancy of all elements in COV, the following steps are included: set sum to the set of the number of all pairwise combinations. in accordance with Calculate all u from matrix U. 3,e The value of .

[0079] Step S15: Display the updated test case redundancy distribution matrix and the coverage redundancy matrix, and evaluate the coverage redundancy of the pairwise combinations of test cases in the test case set.

[0080] In this embodiment, to more intuitively demonstrate the coverage capability and coverage redundancy distribution of the test case set, the results of R are plotted as a curve; where the x-axis is the first row of R, representing specific test cases, and the y-axis is the second row of R, representing the pairwise combination redundancy degree of the current test case relative to the previous i test case sets; and the results of U are plotted as a bar chart. The x-axis is the first row of U, representing pairwise combination coverage redundancy, and the y-axis is the third row of U, representing the proportion of the current pairwise combination coverage redundancy to all pairwise combination sets. Then, based on the results of the curve and bar charts, the coverage redundancy of the pairwise combination test cases in the test case set is evaluated to obtain the evaluation result, and then based on the evaluation result, a referenceable and traceable measure of test case redundancy is provided for optimizing test cases. It can be understood that the combined test case coverage redundancy is defined as follows: the degree to which the test case set TS repeatedly covers the t-way combination when the parameter t-way combination is fully covered by the test case set TS.

[0081] As can be seen, this application obtains a test case set and constructs a corresponding initial test case matrix based on the test case set; re-assigns and encodes the initial test case matrix according to preset rules to obtain an updated target test case matrix, and constructs a parameter combination matrix to characterize the pairwise combination of parameters based on the number of parameters in the test case set; constructs a coverage redundancy matrix to characterize the number of times each pairwise combination of parameters in the test case set is covered based on the parameter value range in the target test case matrix, and constructs a test case redundancy distribution matrix to characterize the pairwise combination coverage redundancy distribution in each test case; updates the coverage redundancy matrix and the test case redundancy distribution matrix using the target test case matrix and the parameter combination matrix, and analyzes and statistically analyzes the elements of the updated coverage redundancy matrix to obtain a coverage redundancy degree matrix to characterize the pairwise combination coverage redundancy ratio; displays the updated test case redundancy distribution matrix and the coverage redundancy degree matrix, and evaluates the coverage redundancy of the pairwise combination test cases in the test case set. Therefore, this application constructs an initial test case matrix based on the obtained test case set, and re-encodes the initial test case matrix to obtain the target test case matrix and constructs a parameter combination matrix to determine the pairwise combination of parameters; constructs a coverage redundancy matrix to determine the number of times each pairwise combination is covered, and constructs a test case redundancy distribution matrix to determine the coverage redundancy distribution of individual test cases in pairwise combinations; then, the coverage redundancy matrix and test case redundancy distribution matrix are updated using the target test case matrix and parameter combination matrix; then, the elements in the updated coverage redundancy matrix are analyzed and statistically analyzed to obtain a coverage redundancy degree matrix, which is used to determine the coverage redundancy ratio of each pairwise combination; finally, the updated test case redundancy distribution matrix and coverage redundancy degree matrix are displayed, and the coverage redundancy of each pairwise combination of test cases in the test case set is evaluated. In this way, the above scheme can quickly determine the coverage capability and coverage redundancy distribution of the test case set in pairwise combination testing, and provide a clear and intuitive display and evaluation, thereby providing a referenceable and traceable measure of test case redundancy for optimizing test cases based on the evaluation results.

[0082] See Figure 4 As shown, this application discloses a specific method for evaluating the redundancy of pairwise combined test case coverage. Compared to the previous embodiment, this embodiment further explains and optimizes the technical solution. Specifically, it includes:

[0083] Step S21: Obtain the test case set and construct the corresponding initial test case matrix based on the test case set.

[0084] Step S22: Determine the number of rows in the initial test case matrix, and based on the number of rows, re-encode the elements of each column in the initial test case matrix in ascending order to obtain the updated target test case matrix, and construct a parameter combination matrix based on the number of parameters in the test case set to characterize the pairwise combination of parameters.

[0085] In this embodiment, a TS' matrix is ​​created to store the re-encoded parameters. This matrix has the same size as the initial test case set matrix. The position within the matrix represents the original parameter's position. Elements in each column of the initial test case matrix are re-encoded in ascending order of row number, i.e., using natural numbers such as 0, 1, 2, 3, 4, 5...m to re-encode the test cases TS, resulting in the updated target test case matrix. The process of re-encoding elements in each column in ascending order of row number to obtain the updated target test case matrix also includes: for each column of the initial test case matrix, determining whether there exists a target element whose original value matches the value of the element to be encoded; if so, assigning the current element to be encoded using the encoded value of the target element; if not, determining the target encoded value of the adjacent encoded element in the current column and assigning the current element to be encoded based on the target encoded value. That is, sequentially reading each row T of the test case set TS matrix. i If i = 1, then let T1 = [0]; if i is not equal to 1, then determine the value of T. i The algorithm checks if the value of a parameter of the current element to be encoded is the same as the value of the same parameter in the previous i elements of T. If they are the same, the encoded value of the same element is used to assign a value to the current element to be encoded; if they are different, the current value is the maximum value of the previous adjacent parameter + 1. This process is repeated until all element values ​​in TS' have been updated. Figure 2 Taking the initial test case matrix as an example, the corresponding target test case matrix TS' can be as follows: Figure 5 As shown.

[0086] Step S23: Based on the pairwise combination of parameters in the parameter combination matrix and the parameter value range in the target test case matrix, determine the target number of rows and columns of the matrix to be constructed. Based on the target number of rows and columns, construct a coverage redundancy matrix to characterize the number of times the pairwise combination of parameters in the test case set is covered, and construct a test case redundancy distribution matrix to characterize the pairwise combination coverage redundancy distribution in each test case.

[0087] In this embodiment, the target number of rows and columns s and q of the matrix COV to be constructed are determined based on the pairwise combination of parameters in the parameter combination matrix and the range of parameter values ​​in the target test case matrix. This represents the combination of two parameters in this test case, that is, a combination of different parameters; q = max(c i,j )*max(c i',j' This is the product of parameters that can take at most two maximum value ranges. Specifically, it is the product of parameters that can take at most two maximum value ranges. Figure 5 Taking the target test case matrix in the example, according to Figure 3 From the parameter combination matrix, we can see that there are 6 possible combinations, meaning the number of rows 's' is 6. Each parameter has a value range of 0 to 3, meaning it has 4 possible value ranges. Therefore, the number of columns 'q' = 4 × 4 = 16. The redundancy coverage matrix is ​​then a 6 × 16 matrix, initialized as follows:

[0088]

[0089] In this matrix, the coordinates are related to the position and value of the parameter in the test case. Each row represents the same meaning as each column in matrix C. For example, in matrix c... 1,1 The current line represents all possible permutations and combinations of the first and second parameters. It's understandable that the first and second parameters both range from 0 to 3, so c... 1,1 This represents the combination of the first value of the first parameter being 0 and the first value of the second parameter being 0; c 1,2 This represents the combination of the first value of the first parameter being 0 and the second value of the second parameter being 1; c 1,16 This represents the combination of the last value 3 of the first parameter and the last value 3 of the second parameter.

[0090] Step S24: Based on each column of the parameter combination matrix, read each row of the target test case matrix sequentially to obtain the reading result. Based on the reading result, update the coverage redundancy matrix and the test case redundancy distribution matrix, and analyze and statistically analyze the elements of the updated coverage redundancy matrix to obtain a coverage redundancy degree matrix used to characterize the coverage redundancy ratio of pairwise combinations.

[0091] In this embodiment, see details. Figure 6As shown in the flowchart, each row of the target test case matrix is ​​read sequentially based on each column of the parameter combination matrix to obtain the reading result. Further, the above-mentioned updating of the coverage redundancy matrix and the test case redundancy distribution matrix based on the reading result includes: determining the corresponding first target position in the coverage redundancy matrix based on the reading result, and increasing the current value of the first target position by a preset unit value to obtain the updated value; if the updated value is greater than the preset unit value, then determining the second target position in the test case redundancy distribution matrix, and increasing the current value of the second target position by the preset unit value. That is, determining the corresponding first target position in the coverage redundancy matrix based on the reading result, and updating the current value of the first target position. Specifically, the preset unit value is 1, that is, if the current position value is 0, then the current position value is +1; if the current position value is not 0, then the current position value is +1, and the corresponding COVER value in the second row of the R matrix is ​​+1. The above steps are repeated until all COV matrices are updated. Figure 3 The parameter combination matrix and Figure 5 Taking the target test case matrix as an example, the updated coverage redundancy matrix and test case redundancy distribution matrix are as follows: Figure 7 and Figure 8 As shown. Figure 7 In the test case set, a value of 1 in the first row and first column indicates that the combination of the first parameter being 0 and the second parameter being 0 has appeared once; a value of 0 in the first row and third column indicates that the combination of the first parameter being 0 and the second parameter being 2 has not appeared. Figure 8 The value in column 5 is 5 and 1, indicating that the fifth test case, compared to the first four test cases, covers only one pairwise combination. In other words, the second test case in the target test case matrix has already shown a combination where the first parameter is 1 and the third parameter is 1. Furthermore, regarding... Figure 7 The coverage redundancy matrix, obtained by analyzing and statistically analyzing the elements of the coverage redundancy matrix, and used to characterize the proportion of coverage redundancy in pairwise combinations, can be represented as follows: Figure 9 As shown.

[0092] Step S25: Display the updated test case redundancy distribution matrix and the coverage redundancy matrix, and evaluate the coverage redundancy of pairwise combinations of test cases in the test case set.

[0093] In this embodiment, Figure 8 The result of matrix R is plotted as follows Figure 10 The graph shown has the x-axis representing the first row of R, indicating specific test cases, and the y-axis representing the second row of R, indicating the pairwise redundancy of the current test case relative to the previous i test case sets. Figure 9The result of matrix U is plotted as follows Figure 11 The bar chart shown has the x-axis representing the first row of U, indicating pairwise combinations covering redundancy, and the y-axis representing the third row of U, indicating the proportion of the current pairwise combination covering redundancy to all pairwise combinations.

[0094] For a more detailed description of the process of step S21, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0095] As can be seen, the above scheme can clearly and intuitively display the coverage capability and coverage redundancy distribution of the test case set in pairwise combined testing using line graphs and bar charts, providing a referenceable and traceable measure of test case redundancy for optimizing test cases.

[0096] See Figure 12 As shown in the figure, this application discloses an evaluation device for redundancy coverage of pairwise combined test cases. The device includes:

[0097] The first matrix construction module 11 is used to obtain a test case set and construct a corresponding initial test case matrix based on the test case set;

[0098] The second matrix construction module 12 is used to re-encode the initial test case matrix according to preset rules to obtain the updated target test case matrix, and to construct a parameter combination matrix based on the number of parameters in the test case set to characterize the pairwise combination of parameters.

[0099] The third matrix construction module 13 is used to construct a coverage redundancy matrix based on the parameter value range in the target test case matrix to characterize the number of times each pair of parameters in the test case set is covered, and to construct a test case redundancy distribution matrix to characterize the coverage redundancy distribution of each pair of parameters in each test case.

[0100] The matrix update statistics module 14 is used to update the coverage redundancy matrix and the test case redundancy distribution matrix using the target test case matrix and the parameter combination matrix, and to analyze and statistically analyze the elements of the updated coverage redundancy matrix to obtain a coverage redundancy degree matrix that characterizes the coverage redundancy ratio of each pair of combinations.

[0101] The coverage redundancy evaluation module 15 is used to display the updated test case redundancy distribution matrix and the coverage redundancy degree matrix, and to evaluate the coverage redundancy of pairwise combined test cases in the test case set.

[0102] As can be seen, this application obtains a test case set and constructs a corresponding initial test case matrix based on the test case set; re-assigns and encodes the initial test case matrix according to preset rules to obtain an updated target test case matrix, and constructs a parameter combination matrix to characterize the pairwise combination of parameters based on the number of parameters in the test case set; constructs a coverage redundancy matrix to characterize the number of times each pairwise combination of parameters in the test case set is covered based on the parameter value range in the target test case matrix, and constructs a test case redundancy distribution matrix to characterize the pairwise combination coverage redundancy distribution in each test case; updates the coverage redundancy matrix and the test case redundancy distribution matrix using the target test case matrix and the parameter combination matrix, and analyzes and statistically analyzes the elements of the updated coverage redundancy matrix to obtain a coverage redundancy degree matrix to characterize the pairwise combination coverage redundancy ratio; displays the updated test case redundancy distribution matrix and the coverage redundancy degree matrix, and evaluates the coverage redundancy of the pairwise combination test cases in the test case set. Therefore, this application constructs an initial test case matrix based on the obtained test case set, and re-encodes the initial test case matrix to obtain the target test case matrix and constructs a parameter combination matrix to determine the pairwise combination of parameters; constructs a coverage redundancy matrix to determine the number of times each pairwise combination is covered, and constructs a test case redundancy distribution matrix to determine the coverage redundancy distribution of individual test cases in pairwise combinations; then, the coverage redundancy matrix and test case redundancy distribution matrix are updated using the target test case matrix and parameter combination matrix; then, the elements in the updated coverage redundancy matrix are analyzed and statistically analyzed to obtain a coverage redundancy degree matrix, which is used to determine the coverage redundancy ratio of each pairwise combination; finally, the updated test case redundancy distribution matrix and coverage redundancy degree matrix are displayed, and the coverage redundancy of each pairwise combination of test cases in the test case set is evaluated. In this way, the above scheme can quickly determine the coverage capability and coverage redundancy distribution of the test case set in pairwise combination testing, and provide a clear and intuitive display and evaluation, thereby providing a referenceable and traceable measure of test case redundancy for optimizing test cases based on the evaluation results.

[0103] Figure 13 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Specifically, it may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement relevant steps in the evaluation method for redundant pairwise test case coverage performed by the electronic device as disclosed in any of the foregoing embodiments.

[0104] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0105] The processor 21 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 21 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 21 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0106] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored on it include operating system 221, computer program 222 and data 223, etc., and the storage method can be temporary storage or permanent storage.

[0107] The operating system 221 manages and controls the various hardware devices and computer programs 222 on the electronic device 20 to enable the processor 21 to perform calculations and processing on the massive amounts of data 223 in the memory 22. The operating system 221 can be Windows, Unix, Linux, etc. The computer program 222, in addition to including a computer program capable of performing the evaluation method for covering redundancy in pairwise test cases executed by the electronic device 20 as disclosed in any of the foregoing embodiments, may further include computer programs capable of performing other specific tasks. The data 223 may include data received by the electronic device from external devices, as well as data collected by its own input / output interface 25.

[0108] Furthermore, embodiments of this application also disclose a computer-readable storage medium storing a computer program. When the computer program is loaded and executed by a processor, it implements the method steps performed in the evaluation process of covering redundancy by pairwise combination of test cases disclosed in any of the foregoing embodiments.

[0109] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0110] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0111] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0112] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0113] The above provides a detailed description of the evaluation method, apparatus, device, and storage medium for pairwise combined test case coverage redundancy provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for evaluating the redundancy of test case coverage in pairwise combinations, characterized in that, include: Obtain the test case set, and construct the corresponding initial test case matrix based on the test case set; The initial test case matrix is ​​re-encoded according to preset rules to obtain the updated target test case matrix, and a parameter combination matrix is ​​constructed based on the number of parameters in the test case set to characterize the pairwise combination of parameters. Based on the parameter value range in the target test case matrix, a coverage redundancy matrix is ​​constructed to characterize the number of times each pair of parameters in the test case set is covered, and a test case redundancy distribution matrix is ​​constructed to characterize the coverage redundancy distribution of each pair of parameters in each test case. The coverage redundancy matrix and the test case redundancy distribution matrix are updated using the target test case matrix and the parameter combination matrix, and the elements of the updated coverage redundancy matrix are analyzed and statistically analyzed to obtain a coverage redundancy degree matrix that characterizes the coverage redundancy ratio of each pair of combinations. The updated test case redundancy distribution matrix and coverage redundancy matrix are displayed, and the coverage redundancy of pairwise combinations of test cases in the test case set is evaluated. The step of re-assigning and encoding the initial test case matrix according to preset rules to obtain the updated target test case matrix includes: Determine the number of rows in the initial test case matrix, and based on the number of rows, re-encode the elements of each column in the initial test case matrix in ascending order to obtain the updated target test case matrix; The process of re-encoding the elements in each column in ascending order based on the row number to obtain the updated target test case matrix also includes: For each column of the initial test case matrix, determine whether there is a target element whose original value is consistent with the value of the element to be encoded. If it exists, then the encoded value of the target element is used to assign a value to the current element to be encoded; If it does not exist, then determine the target encoding value of the encoded element that is adjacent to the current element to be encoded in the current column, and assign a value to the current element to be encoded based on the target encoding value; The construction of a coverage redundancy matrix based on the parameter value ranges in the target test case matrix to characterize the coverage count of pairwise parameter combinations in the test case set includes: The target number of rows and columns of the matrix to be constructed is determined based on the pairwise combination of parameters in the parameter combination matrix and the range of parameter values ​​in the target test case matrix. Based on the target number of rows and columns, a coverage redundancy matrix is ​​constructed to characterize the number of times each pair of parameters in the test case set is covered. The step of updating the coverage redundancy matrix and the test case redundancy distribution matrix using the target test case matrix and the parameter combination matrix includes: Initialize the coverage redundancy matrix and the test case redundancy distribution matrix; The initialized coverage redundancy matrix and test case redundancy distribution matrix are updated using the target test case matrix and the parameter combination matrix.

2. The evaluation method for redundancy coverage of pairwise combined test cases according to claim 1, characterized in that, The step of updating the coverage redundancy matrix and the test case redundancy distribution matrix using the target test case matrix and the parameter combination matrix includes: Based on each column of the parameter combination matrix, each row of the target test case matrix is ​​read sequentially to obtain the reading result; The coverage redundancy matrix and the test case redundancy distribution matrix are updated based on the reading results.

3. The evaluation method for redundancy coverage of pairwise combined test cases according to claim 2, characterized in that, The step of updating the coverage redundancy matrix and the test case redundancy distribution matrix based on the reading results includes: Based on the reading result, the corresponding first target position is determined in the coverage redundancy matrix, and the current value of the first target position is increased by a preset unit value to obtain the updated value; If the updated value is greater than the preset unit value, then the second target position in the test case redundancy distribution matrix is ​​determined, and the current value of the second target position is increased by the preset unit value.

4. An evaluation device for pairwise combined test case coverage redundancy, characterized in that, The method for evaluating the redundancy of pairwise combined test case coverage as described in any one of claims 1 to 3 includes: The first matrix construction module is used to obtain a test case set and construct a corresponding initial test case matrix based on the test case set. The second matrix construction module is used to re-encode the initial test case matrix according to preset rules to obtain the updated target test case matrix, and to construct a parameter combination matrix based on the number of parameters in the test case set to characterize the pairwise combination of parameters. The third matrix construction module is used to construct a coverage redundancy matrix based on the parameter value range in the target test case matrix to characterize the number of times each pair of parameters in the test case set is covered, and to construct a test case redundancy distribution matrix to characterize the coverage redundancy distribution of each pair of parameters in each test case. The matrix update statistics module is used to update the coverage redundancy matrix and the test case redundancy distribution matrix using the target test case matrix and the parameter combination matrix, and to analyze and statistically analyze the elements of the updated coverage redundancy matrix to obtain a coverage redundancy degree matrix that characterizes the coverage redundancy ratio of each pair of combinations. The coverage redundancy evaluation module is used to display the updated test case redundancy distribution matrix and the coverage redundancy degree matrix, and to evaluate the coverage redundancy of pairwise combinations of test cases in the test case set.

5. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the evaluation method for pairwise combined test case coverage redundancy as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, Used to store computer programs; wherein, when the computer programs are executed by a processor, they implement the steps of the evaluation method for covering redundancy in pairwise combined test cases as described in any one of claims 1 to 3.

Images