A test case evaluation method and device, a terminal device and a storage medium

Abstract

Embodiments of the present application disclose a test case evaluation method and device, terminal equipment and a storage medium, wherein the method comprises: obtaining code line numbers executed by at least two test cases to be evaluated, and generating a line number set corresponding to each test case to be evaluated by taking the code line numbers as set elements; classifying the line number sets according to their intersection relations to obtain set types, and obtaining a first score of each line number set according to the set types and a preset classification score table; obtaining a second score of each line number set according to the number of elements of each line number set and the minimum number of elements of line number sets of the same set type; obtaining a third score of each line number set according to the number of parent sets of each line number set and the minimum number of parent sets of line number sets of the same set type; and obtaining an evaluation score of each test case to be evaluated according to the first score, the second score and the third score, thereby solving the technical problem that the relative high and low of the coverage capability of test cases on code cannot be comprehensively reflected in the existing test report.

Description

Technical Field

[0001] This application relates to the field of software testing technology, and in particular to a test case evaluation method, apparatus, terminal device, and storage medium. Background Technology

[0002] With the increasing prevalence of smart devices, the types and quantities of software installed on these devices are also growing. Before software release, it is typically necessary to test various functions of the software based on test cases. A test case is a task description for testing a specific software product, and its content includes test objectives, test environment, input data, test steps, expected results, and test scripts. As software complexity increases, the number and scale of test cases also increase.

[0003] After completing the testing tasks for a specific software based on test cases, a corresponding test report is generated for each test case. The test report records the test results, the scope of code covered, and other information for that test case.

[0004] When the inventors analyzed the test cases in the existing test reports, they found that the multiple test cases used in the testing process could only independently reflect the coverage ability of each test case to the code, or simply compare the intersection of the code coverage of each test case, but could not comprehensively reflect the relative level of the coverage ability of the test cases to the code. Summary of the Invention

[0005] This invention provides a test case evaluation method, apparatus, terminal device, and storage medium, which solves the technical problem that existing test reports cannot comprehensively reflect the relative coverage of test cases to code.

[0006] In a first aspect, embodiments of the present invention provide a test case evaluation method, including:

[0007] Obtain the line numbers of code executed by at least two test cases to be evaluated, and use the line numbers as set elements to generate a set of line numbers corresponding to each test case to be evaluated;

[0008] The row number sets are classified according to their intersection relationship to obtain the set type. The first score of each row number set is obtained according to the set type and the preset classification scoring table.

[0009] The second score for each row number set is obtained based on the number of elements in each row number set and the minimum number of elements in a row number set of the same set type.

[0010] The third score for each row number set is obtained based on the number of parent sets of the row number set and the minimum number of parent sets of the row number sets of the same set type.

[0011] The evaluation score for each use case to be evaluated is obtained based on the first score, the second score, and the third score.

[0012] Secondly, embodiments of the present invention provide a test case evaluation device, comprising:

[0013] The acquisition unit is used to acquire the line numbers of code executed by at least two test cases to be evaluated, and use the line numbers as set elements to generate a set of line numbers corresponding to each test case to be evaluated.

[0014] The first scoring unit is used to classify the row number sets according to the intersection relationship of the row number sets to obtain the set type, and to obtain the first score of each row number set according to the set type and the preset classification scoring table.

[0015] The second scoring unit is used to obtain a second score for each row number set based on the number of elements in each row number set and the minimum number of elements in a row number set of the same set type.

[0016] The third scoring unit is used to obtain the third score for each row number set based on the number of parent sets of the row number set and the minimum number of parent sets of the row number sets of the same set type.

[0017] The evaluation unit is used to obtain the evaluation score for each use case to be evaluated based on the first score, the second score, and the third score.

[0018] Thirdly, embodiments of the present invention provide a terminal device, which includes a processor and a memory;

[0019] The memory is used to store computer programs and transfer them to the processor;

[0020] The processor is used to execute a test case evaluation method, such as the first aspect, according to instructions in a computer program.

[0021] Fourthly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the test case evaluation method as described in the first aspect.

[0022] This application generates a line number set for each test case by analyzing the line numbers executed by the test cases to be evaluated. It then uses mathematical set theory to analyze the intersection of different line number sets, classifying them into set types. A first score is obtained using a pre-defined classification scoring table. A second score is obtained by analyzing the number of elements in each line number set and the minimum number of elements in a line number set of the same set type. A third score is obtained by analyzing the number of parent sets and the minimum number of parent sets in a line number set of the same set type. Clearly, the first, second, and third scores can evaluate the coverage ability of different line number sets for code lines, reflecting the relative coverage ability of test cases. This solves the technical problem that existing test reports cannot comprehensively reflect the relative coverage ability of test cases for code. Attached Figure Description

[0023] Figure 1 A flowchart illustrating a test case evaluation method provided in an embodiment of the present invention;

[0024] Figure 2 This is a test case coverage set diagram provided for embodiments of the present invention;

[0025] Figure 3 This is a schematic diagram of the structure of a test case evaluation device provided in an embodiment of the present invention;

[0026] Figure 4 This is a schematic diagram of the structure of a terminal device provided in an embodiment of the present invention. Detailed Implementation

[0027] The following description and accompanying drawings fully illustrate specific embodiments of this application to enable those skilled in the art to practice them. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. The scope of embodiments of this application includes the entire scope of the claims and all available equivalents of the claims. In this document, each embodiment may be referred to individually or collectively with the term "invention," which is merely for convenience and, if more than one invention is disclosed, is not intended to automatically limit the scope of the application to any single invention or inventive concept. Relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, without requiring or implying any actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, or terminal apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed. The various embodiments in this document are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the structures, products, etc., disclosed in the embodiments, since they correspond to the disclosed parts, the descriptions are relatively simple; relevant details can be found in the method section.

[0028] Example 1

[0029] like Figure 1 As shown, Figure 1 This is a flowchart illustrating a test case evaluation method provided in an embodiment of the present invention. The test case evaluation method provided in this embodiment can be executed by a test case management device, which can be implemented through software and / or hardware. The test case management device can consist of two or more physical entities, or it can consist of a single physical entity. For example, the test case management device can be a computer, a host computer, a tablet, or other devices. The method includes the following steps:

[0030] Step 101: Obtain the line numbers of code executed by at least two test cases to be evaluated, and use the line numbers as set elements to generate a set of line numbers corresponding to each test case to be evaluated.

[0031] In software development, before newly developed software is put into use, its various functions are tested. This requires test cases. However, software often has many functions, so testing a single piece of software typically requires a large number of test cases—e.g., a dozen, several dozen, or even hundreds. Furthermore, software development is a long-term, continuous iterative process. To improve the accuracy of test cases and reduce unnecessary testing, test cases need to be continuously optimized. This application provides testers with a reference for optimizing test cases through a comprehensive evaluation. In this application, the test cases determined before each evaluation are called the test cases to be evaluated. Because this application is based on the relative coverage capabilities of test cases, at least two test cases are required to execute the evaluation process. Before evaluation, the software is operated using the test cases to be evaluated. Different test cases to be evaluated have their own corresponding line numbers. During testing, the JaCoCo tool can be used to collect the line numbers executed by at least two test cases to be evaluated. Then, the line numbers collected during the execution of each test case are used as set elements to generate the line number set corresponding to that test case to be evaluated.

[0032] Step 102: Classify the row number sets according to the intersection relationship to obtain the set type, and obtain the first score of each row number set according to the set type and the preset classification scoring table.

[0033] Different line number sets represent the line numbers of code during the independent execution of the test cases to be evaluated. Depending on the test objectives and test parameters designed for the test cases to be evaluated, the elements (i.e., line numbers) in different line number sets may be completely independent, overlap, partially intersect, contain, or even contain multiple elements. In this application, the relationship between such line number set elements is called an intersection relationship, which specifically includes at least independent existence and intersection, and may also include more granular relationship divisions. For example, if the first line number set includes line numbers 1-200, the second line number set includes line numbers 150-300, and the third line number set includes line numbers 301-350, then the first and second line number sets have an intersection in line numbers 150-200, and the third line number set is completely independent of both the first and second line number sets. From a testing perspective, the desired test cases are those that comprehensively cover all line numbers and execute the same line number as few times as possible. Based on this, and considering the intersection between multiple line number sets, it can be assumed that test cases corresponding to completely independent line number sets are closer to the desired test results, while test cases corresponding to line number sets with more overlapping ranges and / or more overlapping times deviate further from the desired test results.

[0034] This application assigns scores to row number sets of different set types based on their deviation from the expected test results, resulting in a pre-defined classification scoring table. In the classification scoring table, the closer the result is to the expected test result, the higher the corresponding score. Each row number set is classified into a set type based on the intersection relationship between them, and then a first score is obtained for each row number set according to the set type and the pre-defined classification scoring table.

[0035] Step 103: Based on the number of elements in each row number set and the minimum number of elements in a row number set of the same set type, obtain the second score for each row number set.

[0036] Different line number sets of the same set type may contain different numbers of elements. Statistical comparison of the number of elements in each line number set allows for further analysis of the code coverage of each set. To more clearly compare the code coverage capability of each line number set, this application uses the number of elements in the line number set as a scoring reference to evaluate line number sets of the same set type, thereby more accurately reflecting the level of code coverage capability of each line number set. Based on this, a second score can be obtained for each line number set according to the number of elements in the line number set and the minimum number of elements in a line number set of the same set type. For example, if a test case to be evaluated executes code lines 1-200, then the line number set corresponding to this test case includes elements from 1 to 200, and the number of elements in the line number set corresponding to this test case is 200. If another test case executes code lines 400-500, then the line number set corresponding to this test case includes elements from 400-500, and the number of elements in the line number set corresponding to this test case is 100, which is the minimum number of elements in the same set type. Based on the 200 elements in the line number set corresponding to the test case to be evaluated and the 100 minimum elements in the line number set of the same set type, a second score can be obtained for each line number set.

[0037] Step 104: Based on the number of parent sets of the row number set and the minimum number of parent sets of the row number set of the same set type, obtain the third score for each row number set.

[0038] Two line number sets of the same type may have a containment relationship. For example, the first line number set includes line numbers 50-100, and the second line number set includes line numbers 30-150. The second line number set contains the first line number set, and the second line number set is called the parent set of the first line number set. The more parent sets a line number set has, the more test cases there are for overlapping code coverage. Line number sets with more overlapping ranges and / or more overlapping times correspond to test cases that deviate from the expected test results. Therefore, this application uses the number of parent sets as a scoring reference to evaluate line number sets of the same type, thereby more accurately evaluating the code coverage capability of each line number set. Based on this, this application can obtain a third score for each line number set based on the number of parent sets and the minimum number of parent sets for line number sets of the same type. For example, if the parent set of the row number set corresponding to the test case to be evaluated is 3, and the parent set of the row number set corresponding to another test case of the same set type is 0, then the parent set of this test case is the smallest parent set in the same set type. Therefore, based on the parent set of the row number set corresponding to the test case to be evaluated being 3 and the smallest parent set of the row number set of the same set type being 0, the third score of this row number set can be obtained.

[0039] Step 105: Obtain the evaluation score for each use case to be evaluated based on the first score, the second score, and the third score.

[0040] The first score is assigned based on classifying and confirming the type of line number sets. The second and third scores are derived by further evaluating line number sets of the same type. These three scores reflect the evaluation of the code coverage capability of line number sets based on the same classification. Therefore, in order to more comprehensively reflect the code coverage capability of each line number set, the final evaluation score for each test case needs to be obtained by combining the first, second, and third scores.

[0041] In the specific implementation process, step 102 can be achieved through steps 1021 and 1022:

[0042] Step 1021: Classify the row number sets according to the intersection relationship and confirm the corresponding lowest level set type. The level of the set type from high to low is: partially or no intersection, proper subset of other row number sets, equal to the intersection of the other two row number sets, and the sum of the proper subsets of the intersection of the other two row number sets equal to the other row number sets.

[0043] Step 1022: Obtain the first score for each row number set based on the set type and the preset classification scoring table. In the classification scoring table, the higher the level of the set type, the higher the corresponding first score.

[0044] The overlapping or non-overlapping relationships between line number sets are assigned different scores. Line number sets with no overlap receive higher scores, while those with significant overlap receive lower scores. Based on this, this application classifies line number sets according to their intersection relationships, from highest to lowest: partially or no overlap, a proper subset of other line number sets, equal to the intersection of two other line number sets, and the sum of the proper subsets of the intersection of two other line number sets equals the sum of the other line number sets. Assume the five levels are numbered from highest to lowest as Level 1, Level 2, Level 3, Level 4, and Level 5. Line number set A includes line numbers from 10 to 200. If other line number sets do not include line numbers from 10 to 200 or only include a portion of these line numbers, then line number set A belongs to the first level. If other line number sets include line numbers from 5 to 300, which includes line numbers from 10 to 200 in line number set A, then line number set A belongs to the second level. If the intersection of two other line number sets includes line numbers from 10 to 200, which are the same as the line numbers in line number set A, then line number set A belongs to the third level. If the intersection of two other line number sets includes line numbers from 5 to 300, which includes line numbers from 10 to 200 in line number set A, then line number set A belongs to the fourth level. If there are other line number sets that, like line number set A, include line numbers from 10 to 200, then line number set A belongs to the fifth level. Furthermore, a line number set may simultaneously meet two different levels; in this case, the line number set needs to be identified as the set type of the lowest level among the two levels. If line number set A meets both the fourth and fifth levels, then it needs to be identified as the lowest level, namely the fifth level.

[0045] The scores in the pre-defined classification scoring table of this application correspond to the set types. Therefore, the first score of the row number set can be obtained according to the set type and the pre-defined classification scoring table. The higher the level of the row number set, the higher the first score. For example, the score of row number set A, which belongs to the first level, is higher than the scores of other row number sets that belong to the second level.

[0046] Based on the initial score, the base scores for each use case to be evaluated can be determined from highest to lowest as 92 points, 72 points, 50 points, 28 points, and 8 points. Of course, the initial score can also be set to other values; this is merely an example, and those skilled in the art can set it according to the actual situation.

[0047] Of course, in the specific implementation process, step 1021 can also be set in other ways, such as two set types: partial or no intersection, and proper subsets of other row number sets.

[0048] In the specific implementation process, step 103 can be implemented as follows:

[0049] The second score for each row number set is obtained based on the difference between the number of elements in each row number set and the minimum number of elements in a row number set of the same set type.

[0050] Assuming the row number set corresponding to the test case to be evaluated has 200 elements, and the minimum number of elements in the row number set corresponding to the test case of the same set type is 100, then the second score of the row number set of the test case to be evaluated can be obtained based on the difference of 200 and 100, which is 100.

[0051] Based on the scoring of the set type, the line number set of the same set type also has differences. For example, the line number set includes the code line numbers, i.e. the number of elements. The second score is obtained based on the difference between the number of elements in the line number set and the minimum number of elements in the line number set of the same set type. This can realize the subdivision based on the first score.

[0052] In the specific implementation process, step 104 can be implemented as follows:

[0053] The third score for each row number set is obtained based on the difference between the number of parent sets of the row number set and the minimum number of parent sets of the row number set of the same set type.

[0054] Assuming the number of parent sets of the row number set corresponding to the test case to be evaluated is 3, and the minimum number of parent sets of the row number set corresponding to the test case to be evaluated in the same set type is 0, then the third score of the row number set corresponding to the test case to be evaluated can be obtained based on the difference of 3 between 3 and 0.

[0055] Based on the scoring of set types, row number sets of the same set type also differ, such as the number of parent sets of different row number sets. The third score is obtained based on the difference between the number of parent sets of the row number set and the minimum number of parent sets of the row number sets of the same set type, which can realize the subdivision based on the first score.

[0056] In the specific implementation process, step 105 can be implemented as follows:

[0057] The first, second, and third scores are weighted and summed to obtain the evaluation score for each use case to be evaluated.

[0058] It is understandable that the second and third scores are more detailed assessments based on the first score. Therefore, the first score, as the base score, accounts for a relatively large proportion of the overall evaluation score. To balance the different weights of the first, second, and third scores in the evaluation score, this embodiment assigns corresponding weights to the first, second, and third scores, and then performs a weighted sum to obtain the final evaluation score for the use case to be evaluated. In this embodiment, the weight of the first score can be 1, the weight of the second score can be the reciprocal of the maximum number of elements in the row number set of the same set type, and the weight of the third score can be the inverse of the reciprocal of the maximum number of parent sets in the row number set of the same set type. For example, if the maximum number of elements in the row number set of the same set type is 300, then the weight of the second score is 1 / 300; if the maximum number of parent sets in the row number set of the same set type is 4, then the weight of the third score is -1 / 4.

[0059] The calculation process for the evaluation score is as follows:

[0060] S2 = TClen - TClenmin;

[0061] S3 = TCn - TCnmin;

[0062] S = 1*S1 + 1 / a*S2 + (-1 / b)*S3;

[0063] Where S is the evaluation score of the test case to be evaluated, S1 is the first score, S2 is the second score, S3 is the third score, TClen is the number of elements in the row number set, TClenmin is the minimum number of elements in the row number set of the same set type, 1 / a is the reciprocal of the maximum number of elements in the row number set of the same set type, TCn is the number of parent sets of the row number set, TCnmin is the minimum number of parent sets of the row number set of the same set type, and -1 / b is the inverse of the reciprocal of the maximum number of parent sets of the row number set of the same set type.

[0064] In the specific implementation process, after step 105, the following also includes:

[0065] Step 1051: Sort and display the test cases with evaluation scores below the preset threshold from low to high.

[0066] Step 1052: Receive update operations for test cases.

[0067] Generally, test cases with low evaluation scores need to be optimized to improve the accuracy of software testing. Therefore, after evaluating test cases, if the evaluation score is lower than a preset threshold, the test cases need to be sorted and displayed from lowest to highest evaluation score. This allows technical personnel to understand which test cases need optimization and to optimize those with low evaluation scores. Of course, the threshold value in this embodiment can be set to 30 points, 50 points, or other values. In addition, technical personnel can also optimize all test cases and finally receive the updated test cases after optimization, thereby improving the accuracy of software testing.

[0068] During the update process, test cases with lower evaluation scores can be deleted, other test cases can be added, or test cases with the same evaluation scores can be removed to eliminate overlapping test cases.

[0069] Furthermore, after step 1052, the following steps are also included:

[0070] Evaluate the test cases retained after the update operation.

[0071] There are two execution scenarios for evaluating the retained test cases. The first is to evaluate the row number sets corresponding to the test cases based on the original set type. That is, when lower-level test cases are deleted, the level of the row number sets corresponding to the test cases based on the original set type will change. For example, if row number sets A and B were both originally at level 5, after deleting row number set A, there are no row number sets identical to B. At this point, row number set A belongs to the first level, either partially or without overlap. Therefore, when evaluating the row number sets corresponding to the retained test cases after the update operation, the level of row number set B will change, and the evaluation score will also change. The second scenario is to re-evaluate the retained test cases according to steps 101-105. That is, it is necessary to return to step 101 to re-evaluate the test cases, obtain the first, second, and third scores again, and then obtain the final evaluation score.

[0072] This application generates a line number set for each test case by analyzing the line numbers executed by the test cases to be evaluated. It then uses mathematical set theory to analyze the intersection of different line number sets, classifying them into set types. A first score is obtained using a pre-defined classification scoring table. A second score is obtained by analyzing the number of elements in each line number set and the minimum number of elements in a line number set of the same set type. A third score is obtained by analyzing the number of parent sets and the minimum number of parent sets in a line number set of the same set type. Clearly, the first, second, and third scores can evaluate the coverage ability of different line number sets for code lines, reflecting the relative coverage ability of test cases. This solves the technical problem that existing test reports cannot comprehensively reflect the relative coverage ability of test cases for code.

[0073] To more clearly illustrate the technical solutions of the embodiments of this application, the following description is provided in conjunction with specific application scenarios:

[0074] Please see Figure 2 Suppose there are 7 test cases to be evaluated, and the line number sets corresponding to these 7 test cases are A, B, C, D, E, F, and G. The line numbers covered by line number sets A to G are 1-200, 1-200, 150-400, 250-300, 160-180, 150-200, and 150-200, respectively. Clearly, line number sets A and B both cover line numbers 1-200 and belong to the same set; line number set C covers line numbers 150-400. That is, the intersection of line number sets A (or B) and C is the area they both cover. Line numbers 150-200 represent the intersection of test case coverage, represented by line number set F. Line number set G and line number set F are the same set, covering line numbers 150-200. Line number set E covers line numbers 160-180. Line number set F(G) covers line numbers that include those covered by line number set E, and line number set F(G) ≠ line number set E. Therefore, line number set E is a proper subset of line number set F(G). Line number set D covers line numbers 250-300. Line number set C contains set D, and line number set C ≠ line number set D. Therefore, line number set D is a proper subset of line number set C.

[0075] The row number sets are classified into set types, from highest to lowest: Level 1 (partial or no intersection), Level 2 (proper subsets of other row number sets), Level 3 (equal to the intersection of the other two row number sets), Level 4 (proper subsets of the intersection of the other two row number sets), and Level 5 (equal to other row number sets). Row number sets A and B belong to both Level 1 and Level 5, so they are classified as Level 5; row number set C belongs to Level 1; row number set D belongs to Level 2; row number set E belongs to both Level 2 and Level 4, so it is classified as Level 4; row number sets F and G belong to Level 2, Level 3, and Level 5, so they are classified as Level 5.

[0076] The preset classification scoring table has the following scores: 92, 72, 50, 28, and 8. Specifically, the first level has a score of 92, the second level has a score of 72, the third level has a score of 50, the fourth level has a score of 28, and the fifth level has a score of 8. Therefore, based on the set type and the preset classification scoring table, the first scores for row number sets A to G are 8, 8, 92, 72, 28, 8, and 8, respectively.

[0077] Regarding the second score, let's take row number sets A, B, F, and G as examples for specific evaluation. Row number sets B, F, and G belong to the same set type as row number set A. Row number sets A and B have 200 elements, while row number sets F and G have 20 elements. Therefore, the second score for row number set A is the difference between its element count of 200 and the minimum element count of 20 for a row number set of the same type, which is 180. Similarly, the second score for row number set B is also 180. The second score for row number set F is the difference between its element count of 20 and the minimum element count of 20 for a row number set of the same type, which is 0. Similarly, the second score for row number set G is 0. The weight corresponding to row number set A (B) is 1 / 200, and the weighted result of the second score is 0.9. The weight corresponding to row number set F (G) is 1 / 200, and the weighted result of the second score is 0.

[0078] Row number set F belongs to the same set type as row number set F. Row number sets F and G have 2 parent sets, while row number sets A and B have 0 parent sets. Therefore, the third score of row number set F is the difference of 2 between the number of parent sets F (2) and the smallest parent set of row number sets of the same set type (0). Similarly, the third score of row number set G is 2, and the third score of row number sets A and B is 0. The weight corresponding to row number set A (B) is -1 / 2, and the weighted result of its third score is 0. The weight corresponding to row number set F (G) is -1 / 2, and the weighted result of its third score is -1 / 4.

[0079] Finally, the evaluation score of the test case to be rated corresponding to the row number set A(B) is the sum of the first score of 8 points, the weighted result of the second score of 0.9, and the weighted result of the third score of 0, which is 8.9; the evaluation score of the test case to be rated corresponding to the row number set F(G) is the sum of the first score of 8 points, the weighted result of the second score of 0, and the weighted result of the third score of -1 / 4, which is 7.75.

[0080] It should be noted that, based on Figure 2 In the exemplary description, the division of the set type and the specific numerical limitation of the corresponding scores are merely illustrative. For example, the scores for the five levels are 92, 72, 50, 28, and 8 points, respectively. These scores can also be set to other values, such as 10, 8, 6, 4, and 2 points. Overall, through the weighted calculation of the second and third scores, the first score may vary from less than 1 to greater than -1. Therefore, the first score between two adjacent levels should maintain a difference of at least 2 points.

[0081] Example 2

[0082] , Figure 3 This is a schematic diagram of a test case evaluation device provided in an embodiment of the present invention, with reference to... Figure 3 The test case evaluation device includes an acquisition unit 201, a first scoring unit 202, a second scoring unit 203, a third scoring unit 204, and an evaluation unit 205.

[0083] The system includes: an acquisition unit 201, which acquires the line numbers of at least two test cases to be evaluated, and uses these line numbers as set elements to generate a line number set for each test case to be evaluated; a first scoring unit 202, which classifies the line number sets according to their intersection to obtain a set type, and obtains a first score for each line number set based on the set type and a preset classification scoring table; a second scoring unit 203, which obtains a second score for each line number set based on the number of elements in each line number set and the minimum number of elements in a line number set of the same set type; a third scoring unit 204, which obtains a third score for each line number set based on the number of parent sets of the line number set and the minimum number of parent sets in a line number set of the same set type; and an evaluation unit 205, which obtains an evaluation score for each test case to be evaluated based on the first, second, and third scores.

[0084] Correspondingly, the first scoring unit 202 includes:

[0085] The set classification module is used to classify and identify the lowest-level set type of the row number sets according to their intersection relationship. The levels of the set types from high to low are: partially or no intersection, proper subset of other row number sets, equal to the intersection of the other two row number sets, and the sum of the proper subsets of the intersection of the other two row number sets equals the other row number sets.

[0086] The classification and scoring module is used to obtain the first score for each row number set based on the set type and the preset classification and scoring table. In the classification and scoring table, the higher the level of the set type, the higher the corresponding first score.

[0087] Correspondingly, the second scoring unit 203 is specifically used to: obtain a second score for each row number set based on the difference between the number of elements in each row number set and the minimum number of elements in a row number set of the same set type.

[0088] Correspondingly, the third scoring unit 204 is specifically used for:

[0089] The third score for each row number set is obtained based on the difference between the number of parent sets of the row number set and the minimum number of parent sets of the row number set of the same set type.

[0090] Correspondingly, evaluation unit 205 is specifically used for:

[0091] The first, second, and third scores are weighted and summed to obtain the evaluation score for each use case to be evaluated.

[0092] The weight of the first score is 1, the weight of the second score is the reciprocal of the maximum number of elements in the row number set of the same set type, and the weight of the third score is the inverse of the reciprocal of the maximum number of parent sets in the row number set of the same set type.

[0093] Based on the above embodiments, the test case evaluation device further includes:

[0094] The sorting unit is used to sort and display test cases whose evaluation scores are lower than a preset threshold from lowest to highest.

[0095] The update unit is used to receive update operations for test cases.

[0096] The test case evaluation device provided in this embodiment of the invention is contained in an electronic device and can be used to execute the corresponding test case evaluation method provided in the above embodiment, and has corresponding functions and beneficial effects.

[0097] It is worth noting that in the embodiments of the test case evaluation device described above, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0098] Example 3

[0099] Figure 4 This is a schematic diagram of the structure of a terminal device provided in an embodiment of the present invention, such as... Figure 4As shown, a terminal device 30 includes a processor 300 and a memory 301.

[0100] The memory 301 is used to store the computer program 302 and to transfer the computer program 302 to the processor;

[0101] The processor 300 is used to execute the steps in one embodiment of the test case evaluation method described above, according to the instructions in the computer program 302.

[0102] For example, the computer program 302 may be divided into one or more modules / units, one or more of which are stored in the memory 301 and executed by the processor 300 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 302 in the terminal device 30.

[0103] Terminal device 30 can be a computing device such as a desktop computer, laptop, handheld computer, or cloud server. Terminal device 30 may include, but is not limited to, processor 300 and memory 301. Those skilled in the art will understand that... Figure 4 This is merely an example of terminal device 30 and does not constitute a limitation on terminal device 30. It may include more or fewer components than shown, or combine certain components, or different components. For example, terminal device 30 may also include input / output devices, network access devices, buses, etc.

[0104] The processor 300 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0105] The memory 301 can be an internal storage unit of the terminal device 30, such as a hard disk or memory of the terminal device 30. The memory 301 can also be an external storage terminal device of the terminal device 30, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the terminal device 30. Furthermore, the memory 301 can include both internal and external storage units of the terminal device 30. The memory 301 is used to store computer programs and other programs and data required by the terminal device 30. The memory 301 can also be used to temporarily store data that has been output or will be output.

[0106] The aforementioned terminal device includes a file transfer device, which can be used to execute test case evaluation methods and has corresponding functions and beneficial effects.

[0107] Example 4

[0108] This invention also provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program performs relevant operations in the test case evaluation method provided in any embodiment of this application, and has corresponding functions and beneficial effects.

[0109] Those skilled in the art will understand that embodiments of this application may be provided as methods, systems, or computer program products.

[0110] Therefore, this application may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, produce implementations of the flowchart... Figure 1 One or more processes and / or boxes Figure 1The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0111] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory. Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0112] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0113] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0114] Note that the above are merely preferred embodiments and the technical principles applied in this invention. Those skilled in the art will understand that the embodiments of this invention are not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the protection scope of this invention. Therefore, although the embodiments of this invention have been described in detail above, the embodiments of this invention are not limited to the above embodiments. More other equivalent embodiments may be included without departing from the concept of the embodiments of this invention, and the scope of the embodiments of this invention is determined by the scope of the appended claims.

Claims

1. A test case evaluation method, characterized in that, include: Obtain the line numbers of code executed by at least two test cases to be evaluated, and use the line numbers as set elements to generate a set of line numbers corresponding to each test case to be evaluated; The row number sets are classified according to their intersection relationship to obtain a set type, and a first score for each row number set is obtained according to the set type and a preset classification scoring table. A second score is obtained for each row number set based on the number of elements in each row number set and the minimum number of elements in a row number set of the same set type. The third score for each row number set is obtained based on the number of parent sets of the row number set and the minimum number of parent sets of the row number sets of the same set type. The evaluation score for each use case to be evaluated is obtained based on the first score, the second score, and the third score. Test cases with evaluation scores below a preset threshold are sorted and displayed in ascending order of evaluation scores; Receive update operations for test cases; Evaluate the test cases retained after the update operation; The step of classifying the row number sets according to their intersection relationships to obtain set types, and obtaining a first score for each row number set based on the set types and a preset classification scoring table, includes: The row number sets are classified according to their intersection relationships to determine the lowest level of the set type. The levels of the set types from high to low are: partially or no intersection, proper subset of other row number sets, equal intersection with two other row number sets, and the sum of the proper subsets of the intersection of two other row number sets equal to the other row number sets. The first score of each row number set is obtained according to the set type and the preset classification scoring table. In the classification scoring table, the higher the level of the set type, the higher the corresponding first score. Among them, the test cases corresponding to completely independent row number sets are closer to the expected test results, while the test cases corresponding to row number sets with more overlapping ranges and / or more overlapping times deviate more from the expected test results. The preset classification scoring table assigns scores to row number sets of different set types according to the degree of deviation from the expected test results. The closer to the expected test results, the higher the corresponding score.

2. The test case evaluation method according to claim 1, characterized in that, The second score for each row number set is obtained based on the number of elements in each row number set and the minimum number of elements in a row number set of the same set type, including: A second score is obtained for each row number set based on the difference between the number of elements in each row number set and the minimum number of elements in a row number set of the same set type.

3. The test case evaluation method according to claim 1, characterized in that, The third score for each row number set is obtained based on the number of parent sets of the row number set and the minimum number of parent sets of row number sets of the same set type, including: The third score for each row number set is obtained based on the difference between the number of parent sets of the row number set and the minimum number of parent sets of the row number set of the same set type.

4. The test case evaluation method according to claim 1, characterized in that, The step of obtaining the evaluation score for each use case to be evaluated based on the first score, the second score, and the third score includes: The first score, the second score, and the third score are weighted and summed to obtain the evaluation score for each use case to be evaluated.

5. The test case evaluation method according to claim 4, characterized in that, The weight of the first score is 1, the weight of the second score is the reciprocal of the maximum number of elements in the row number set of the same set type, and the weight of the third score is the inverse of the reciprocal of the maximum number of parent sets in the row number set of the same set type.

6. A test case evaluation device, characterized in that, include: The acquisition unit is used to acquire the line numbers of code executed by at least two test cases to be evaluated, and use the line numbers as set elements to generate a set of line numbers corresponding to each test case to be evaluated. The first scoring unit is used to classify the row number sets according to the intersection relationship of the row number sets to obtain the set type, and to obtain the first score of each row number set according to the set type and the preset classification scoring table; The second scoring unit is used to obtain a second score for each row number set based on the number of elements in each row number set and the minimum number of elements in a row number set of the same set type. The third scoring unit is used to obtain a third score for each row number set based on the number of parent sets of the row number set and the minimum number of parent sets of row number sets of the same set type. An evaluation unit is used to obtain an evaluation score for each of the test cases to be evaluated based on the first score, the second score, and the third score. The sorting unit is used to sort and display test cases with evaluation scores below a preset threshold from lowest to highest. The update unit is used to receive update operations for test cases; Evaluate the test cases retained after the update operation; The first scoring unit includes: The set classification module is used to classify the row number sets according to the intersection relationship of the row number sets to determine the corresponding lowest level set type. The level of the set type from high to low is: partially or no intersection, proper subset of other row number sets, equal to the intersection of the other two row number sets, and the sum of the proper subsets of the intersection of the other two row number sets equals the other row number sets. The classification and scoring module is used to obtain the first score of each row number set according to the set type and the preset classification and scoring table. In the classification and scoring table, the higher the level of the set type, the higher the corresponding first score. Among them, the test cases corresponding to completely independent row number sets are closer to the expected test results, while the test cases corresponding to row number sets with more overlapping ranges and / or more overlapping times deviate more from the expected test results. The preset classification scoring table assigns scores to row number sets of different set types according to the degree of deviation from the expected test results. The closer to the expected test results, the higher the corresponding score.

7. A terminal device, characterized in that, The terminal device includes a processor and a memory; The memory is used to store computer programs and to transfer the computer programs to the processor; The processor is configured to execute a test case evaluation method as described in any one of claims 1-5 according to instructions in the computer program.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the test case evaluation method as described in any one of claims 1-5.

Images