Test case quality classification and scoring method and device and electronic equipment

Abstract

The invention discloses a test case quality classification and scoring method and device and electronic equipment, and belongs to the technical field of test case quality evaluation.The method comprises the steps that the scoring grade of a test case is determined, and the grade label of the test case is determined according to the scoring grade; setting a quality label distribution rule according to the grade labels; on the basis of the grade labels, sequentially carrying out quality label distribution on the test cases according to quality label distribution rules so as to generate quality label training data sets with the same quantity as the quality label distribution rules; and according to the quality label training data set, constructing a corresponding number of quality classification models taking the quality labels as classification basis, according to the quality classification models, determining the quality classification probability of the to-be-evaluated test cases, and according to the quality classification probability, determining the quality classification of the to-be-evaluated test cases. According to the method, the grade label probability is further calculated by calculating the quality classification probability, so that accurate and multi-grade quality classification and scoring are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of test case quality assessment, and in particular to a test case quality classification and scoring method, device and electronic equipment. Background Art

[0002] Software testing is a necessary process to ensure that software products meet functional requirements, are defect-free, and maintain high performance and security. With the continuous development of large language models, large language models have gradually begun to be applied to the field of software testing, which can greatly improve the speed and number of test case generation. The model can comprehensively consider a large number of code instances and historical data, and the generated test cases can more comprehensively cover potential risk points. However, for large language models, the test cases they generate may not be completely correct, and for some specific business logic and edge cases, the model cannot fully understand the requirements behind them, resulting in the generated test cases not fully meeting actual needs.

[0003] In the prior art, large language models are combined with deep learning to generate test cases and perform quality assessment on test cases. Most current assessment models use a simple binary classification system, which divides test cases into "qualified" or "unqualified". It is impossible to accurately evaluate the quality of software test cases by multi-level division to describe the quality level of test cases in detail.

[0004] Therefore, there is an urgent need for a method for test case quality classification and scoring, which can accurately and multi-level quality classification of test cases and then accurately score the quality of test cases. Summary of the invention

[0005] In view of this, it is necessary to provide a test case quality classification and scoring method, device and electronic device, which can accurately and multi-level quality classification of test cases and then accurately score the quality of test cases.

[0006] In order to solve the above technical problems, on the one hand, the present invention provides a test case quality classification, including: Determine a rating level for the test case, and determine a level label for the test case according to the rating level; Setting a quality label allocation rule according to the grade label, wherein the number of the grade labels is one more than the number of the quality label allocation rules; Based on the grade labels, assigning quality labels to the test cases in turn according to the quality label assignment rules to generate a quality label training data set with the same number as the quality label assignment rules; A corresponding number of quality classification models based on the quality labels are constructed according to the quality label training data set, the quality classification probabilities of the test cases to be evaluated are determined according to the quality classification models, and the quality classifications of the test cases to be evaluated are determined according to the quality classification probabilities.

[0007] In a possible implementation, determining a rating level of a test case, and determining a level label of the test case according to the rating level includes: Conduct quality assessment of test cases based on the preset software test management tool and determine the test case scoring level; Map the test cases to the grade labels according to the rating levels to generate grade labels for the test cases.

[0008] In a possible implementation, setting a quality label assignment rule according to the grade label, and sequentially using the quality label assignment rule to assign quality labels to the test cases to generate a quality label training data set with the same number as the quality label assignment rule, including: Set the number of level labels to N; According to the N grade labels, set N-1 quality label allocation rules; According to the N-1 quality label assignment rules, test cases with labels greater than i in the test cases are assigned positive quality labels, and test cases with level labels less than or equal to i are assigned negative quality labels. , and are all positive integers greater than 1, generating A quality labeled training dataset.

[0009] In a possible implementation, a corresponding number of quality classification models based on the quality labels are constructed according to the quality label training data set, quality classification probabilities of the test cases to be evaluated are determined according to the quality classification models, and quality classifications of the test cases to be evaluated are determined according to the quality classification probabilities, including: According to the The CodeT5 model is iteratively trained on each quality label training dataset to determine the negative quality classification probability of the test cases in each training dataset. According to the negative quality classification probability, the positive quality classification probability is calculated, and according to the Quality label training dataset and its positive quality classification probability construction A quality classification model; According to the quality classification models, and determine the test cases to be evaluated in turn. Quality classification probability values; According to the The quality classification probability value determines the quality classification of the test case to be evaluated.

[0010] In a second aspect, the present invention further provides a test case scoring method, comprising: Determine the probability of each quality classification of the test case to be evaluated based on the test case quality classification method; Constructing an objective function of the grade label probability according to the quality classification probability, calculating the probability of each grade label of the test case to be evaluated according to the objective function and the quality classification probability, and determining the grade label of the test case to be evaluated according to the maximum value of the grade label probability; Among them, the test case quality classification method is any one of the above-mentioned test case quality classification methods.

[0011] In one possible implementation, a target function of the class label probability is constructed based on the quality classification probability. , in is the number of level labels, is the level label, and is a positive integer greater than 1, is the probability value of the level label, that is, the level label of the test case is The probability of is the quality classification probability, that is, the test case grade label is greater than probability.

[0012] In a possible implementation, calculating the probability of each level label of the test case to be evaluated according to the objective function and the quality classification probability, and determining the level label of the test case to be evaluated according to the maximum value of the level label probability includes: According to the grade label probability objective function and the quality classification probability, the grade label probabilities of the test cases to be evaluated from 1 to N are calculated respectively; According to the level label probability, the level label corresponding to the maximum level label probability is selected as the level label of the test case to be evaluated.

[0013] In a third aspect, the present invention further provides a test case quality classification device, comprising: A grade label determination module determines the grade of the test case and determines the grade label of the test case according to the grade; A quality label allocation rule generation module is configured to set a quality label allocation rule according to the grade label, wherein the number of the grade label is one more than the number of the quality label allocation rule; A quality label training data set generation module, based on the grade labels, sequentially uses the quality label assignment rules to assign quality labels to the test cases to generate the same number of quality label training data sets as the quality label assignment rules; A quality classification module constructs a corresponding number of quality classification models based on the quality labels according to the quality label training data set, determines the quality classification probability of the test case to be evaluated according to the quality classification model, and determines the quality classification of the test case to be evaluated according to the quality classification probability.

[0014] In a fourth aspect, the present invention further provides a test case scoring device, comprising: A quality classification probability generation module determines the quality classification probabilities of the test cases to be evaluated based on the test case quality classification method; The scoring module constructs an objective function of the grade label probability according to the quality classification probability, calculates the probability of each grade label of the test case to be evaluated according to the objective function and the quality classification probability, and determines the grade label of the test case to be evaluated according to the maximum value of the grade label probability.

[0015] In a fifth aspect, the present invention also provides an electronic device comprising a memory and a processor, wherein the memory is used to store programs and data; the processor is coupled to the memory, and is used to execute the program stored in the memory, to implement the test case quality classification method as described above, and / or to implement the test case scoring method as described above.

[0016] The beneficial effects of the present invention are as follows: first, the grade label of the test case is determined according to the scoring grade of the test case, and multiple quality label allocation rules are set according to the grade label, and the number of grade labels is one more than the number of quality label allocation rules; then, the quality labels are allocated to the test cases according to the quality label allocation rules in turn, and the same number of quality label training data sets as the quality label allocation rules are generated; finally, the same number of quality classification models with quality labels as the classification basis are respectively constructed according to multiple quality label training data sets, and the quality classification probability of the test case to be evaluated is determined in turn according to the multiple quality classification models, and the quality classification of the test case to be evaluated is determined according to each quality classification probability. At the same time, the quality classification probabilities of the test case to be evaluated are determined based on the quality classification method; the objective function of the grade label probability is constructed according to the quality classification probability, and the grade label probabilities of the test case to be evaluated are calculated in turn according to the objective function and the quality classification probability, and the grade label corresponding to the maximum value of the grade label probability is selected as the grade label of the test case to be evaluated. The present invention can allocate multiple grade labels to test cases according to actual test case application needs, and then allocate quality labels to the test cases in turn according to the grade labels, generate multiple quality label training data sets, the number of quality label training data sets is one less than the number of grade labels, and a quality classification probability model is constructed according to the quality label training data set to determine the quality classification probability of the test case to be evaluated that is higher than each scoring level, and classify the test case to be evaluated according to the quality classification probability. At the same time, the probability of each grade label is calculated according to the quality classification probability, and the scoring level with the largest grade label probability is screened out as the grade label of the test case to be evaluated, thereby realizing accurate and multi-level quality classification and scoring of the test case, and providing an accurate basis for the application of the test case. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 A flow chart of an embodiment of the test case quality classification method provided by the present invention; Figure 2 For the present invention Figure 1 A schematic flow chart of an embodiment of step S101; Figure 3 For the present invention Figure 1 A schematic flow chart of an embodiment of step S104; Figure 4A schematic diagram of a flow chart of an embodiment of the test case scoring method provided by the present invention; Figure 5 For the present invention Figure 4 A schematic diagram of a flow chart of an embodiment of step S402; Figure 6 A schematic diagram of the structure of an embodiment of the test case quality classification device provided by the present invention; Figure 7 A schematic diagram of the structure of an embodiment of the test case scoring device provided by the present invention; Figure 8 A schematic diagram of the structure of an embodiment of the test case quality classification and scoring electronic device provided by the present invention. DETAILED DESCRIPTION

[0019] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0020] In the description of the embodiments of the present invention, unless otherwise specified, "multiple" means two or more than two. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" may mean: A exists alone, A and B exist at the same time, and B exists alone.

[0021] The descriptions of "first", "second", etc. involved in the embodiments of the present invention are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the technical features defined as "first" or "second" may explicitly or implicitly include at least one of the features.

[0022] Reference to an "embodiment" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiment may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0023] Before describing the embodiments, the following definitions are given for the relevant terms: The CodeT5 model is a pre-trained encoder-decoder model with large-scale training, multi-task learning, format preservation, cross-language capabilities, and identifier awareness for code generation, code repair and optimization, code search language conversion, etc. ‌‌

[0024] The present invention provides a test case quality classification method, device and electronic equipment, which are described below respectively.

[0025] Figure 1 A flow chart of an embodiment of the test case quality classification method provided by the present invention is as follows: Figure 1 As shown, the test case quality classification methods include: S101, determining a rating level of the test case, and determining a level label of the test case according to the rating level; S102, setting a quality label allocation rule according to the grade label, wherein the number of grade labels is one more than the number of quality label allocation rules; S103, based on the grade labels, assigning quality labels to the test cases in accordance with the quality label assignment rules in turn, so as to generate a quality label training data set with the same number as the quality label assignment rules; S104. Construct a corresponding number of quality classification models based on quality labels according to the quality label training data set, determine the quality classification probability of the test case to be evaluated according to the quality classification model, and determine the quality classification of the test case to be evaluated according to the quality classification probability.

[0026] It should be noted that based on PCA (principal component analysis) or cluster analysis, according to the test management tool combined with the actual historical test data, the test case quality is evaluated according to the compilability, correctness, readability and maintainability standards of the test cases, and the test case scoring level is determined. The test case scoring level can be divided into multiple levels according to actual needs, including excellent, good, average and poor.

[0027] It should be further explained that the grade label of the test case is assigned according to the scoring level of the test case. The grade label is a positive integer. The larger the label value, the better the quality of the test case. In this embodiment, the best 4 grade labels are selected. The mapping relationship between the scoring level and the grade label is that excellent corresponds to grade label 4, good corresponds to grade label 3, general corresponds to grade label 2, and poor corresponds to grade label 1.

[0028] It should be further explained that the number of grade labels is one more than the number of quality label allocation rules. When there are 4 grade labels, there are 3 quality label allocation rules, namely, the quality label allocation rule when it is greater than 3, the quality label allocation rule when it is greater than 2, and the quality label allocation rule when it is greater than 1.

[0029] It should be further explained that the test cases in the training data set and the evaluation test cases are automatically generated based on large language models such as StarCoder to ensure the consistency of the input.

[0030] This embodiment determines the grade label of the test case according to the scoring grade of the test case, sets multiple quality label allocation rules according to the grade label, and the number of grade labels is one more than the number of quality label allocation rules; then, the quality labels are allocated to the test cases according to the quality label allocation rules in turn, and the same number of quality label training data sets as the quality label allocation rules are generated; finally, the same number of quality classification models with quality labels as the classification basis are respectively constructed according to the multiple quality label training data sets, and the quality classification probability of the test case to be evaluated is determined in turn according to the multiple quality classification models, and the quality classification of the test case to be evaluated is determined according to each quality classification probability. This embodiment can allocate multiple grade labels to the test case according to the actual test case application needs, and then the quality labels are allocated to the test case in turn according to the grade labels, and multiple quality label training data sets are generated, and the number of quality label training data sets is one less than the number of grade labels, and a quality classification probability model is constructed according to the quality label training data set to determine the quality classification probability of the test case to be evaluated that is higher than each scoring grade, and the test case to be evaluated is classified according to the quality classification probability, thereby realizing accurate and multi-level quality classification of the test case, providing an accurate basis for the application of the test case.

[0031] This embodiment assigns quality labels to test cases according to scoring levels, determines quality classification probabilities based on quality labels, and classifies test cases according to quality classification probabilities, thereby achieving accurate and multi-level quality classification of test cases and providing an accurate basis for the application of test cases.

[0032] In some embodiments of the present invention, Figure 2 As shown, Figure 2 The present invention provides Figure 1 The flowchart of step S101 in an embodiment includes: S201. Perform quality assessment on test cases based on a preset software test management tool to determine a test case scoring level; S202: Map the test cases to the grade labels according to the rating grades, and generate grade labels for the test cases.

[0033] 1. It should be noted that historical test data is automatically generated based on large language models such as StarCoder, and multiple test cases are generated for each historical tested code. Based on PCA (principal component analysis) or cluster analysis, software test management tools such as Selenium software test management tools and TestRail software test management tools are used. In combination with actual historical test data, historical training test cases are evaluated for quality based on the compilability, correctness, readability, maintainability and other standards of the test cases, and the scoring levels of the training test cases are generated. For example, for m historical tested codes, a large language model is used, and z training test cases are generated for each historical tested code. All test cases are evaluated for quality, and a grade label is assigned to each test case according to the scoring level, to obtain data set A: , , Among them, A is the historical test case data set, m is the number of historical tested codes, and z is the number of test cases for each historical tested code. is the i-th historical tested code, is the jth historical test case of the i-th historical tested code.

[0034] In some embodiments of the present invention, quality label assignment rules are set according to grade labels, and quality labels are assigned to test cases using the quality label assignment rules in sequence to generate a quality label training data set with the same number as the quality label assignment rules, including: Set the number of level labels to N; According to N level labels, set N-1 quality label allocation rules; According to the N-1 quality label assignment rules, the test cases with labels greater than i are assigned positive quality labels, and the test cases with level labels less than or equal to i are assigned negative quality labels. , and are all positive integers greater than 1, generating A quality labeled training dataset.

[0035] Specifically, in this embodiment, the number of level labels is set to 4, and 3 quality label allocation rules are set according to the 4 level labels. All test cases with a level label greater than 1 in multiple test cases are assigned a positive quality label, and all test cases with a level label less than or equal to 1 in multiple test cases are assigned a negative quality label to generate a first training data set; all test cases with a level label greater than 2 in multiple test cases are assigned a positive quality label, and all test cases with a level label less than or equal to 2 in multiple test cases are assigned a negative quality label to generate a second training data set; all test cases with a level label greater than 3 in multiple test cases are assigned a positive quality label, and all test cases with a level label less than or equal to 3 in multiple test cases are assigned a negative quality label to generate a third training data set; for example, data set A is assigned quality labels in sequence to obtain the following 3 data sets: , , : , , , , Among them, A is the test case data set, m is the number of tested codes, and z is the number of test cases for each tested code. , , are three quality label training data sets, m is the number of tested codes, z is the number of test cases for each tested code, is a positive quality label, is a negative quality label.

[0036] In this embodiment, according to the level labels of the test cases, quality labels are assigned to the test cases according to rules that are higher than the level labels, so as to obtain a quality label training data set.

[0037] In some embodiments of the present invention, Figure 3 As shown, Figure 3 The present invention provides Figure 1 The flowchart of step S103 of an embodiment includes: S301, according to The CodeT5 model is iteratively trained on each quality label training dataset to determine the negative quality classification probability of the test cases in each training dataset. S302, based on the negative quality classification probability, calculate the positive quality classification probability, according to Quality label training dataset and its positive quality classification probability construction A quality classification model; S303, according to quality classification models, and determine the test cases to be evaluated in turn. Quality classification probability values; S304, according to The quality classification probability value determines the quality classification of the test case to be evaluated.

[0038] Specifically, according to the generated , , The three quality label training data sets are used to iteratively train the CodeT5 model, and the probabilities of negative quality labels in the three quality label training data sets are obtained in turn. The probability of positive quality classification is determined according to the probability value of the negative quality label. For example, The negative quality classification probability of the quality label training data set is the probability that the grade label is 1. The positive quality classification probability, that is, the quality classification probability with a grade label greater than 1, is obtained by conversion, and a quality classification model with a grade label greater than 1 is constructed. The negative quality classification probability of the quality label training data set is the sum of the probabilities of a score of 1 and a score of 2. The positive quality classification probability, that is, the probability of a grade label greater than 2, is obtained by conversion, and a quality classification model with a grade label greater than 2 is constructed; The negative quality classification probability of the labeled training data set is the sum of the probabilities of scores 1, 2, and 3. The positive quality classification probability, that is, the probability that the grade label is greater than 3, is obtained by conversion, and a quality classification model with grade labels higher than 3 is constructed.

[0039] This embodiment iteratively trains the CodeT5 model according to different quality label training data sets to construct a quality classification model, which can accurately calculate the quality classification probability of each quality level of the test case, thereby enabling accurate and multi-level quality classification of the test case.

[0040] In order to better implement the test case quality classification method in the embodiment of the present invention, based on the test case quality classification method, correspondingly, Figure 4 As shown, the embodiment of the present invention also provides a test case scoring method, including: S401, determining the quality classification probabilities of the test cases to be evaluated based on the test case quality classification method; S402, constructing an objective function of the grade label probability according to the quality classification probability, calculating the probability of each grade label of the test case to be evaluated according to the objective function and the quality classification probability, and determining the grade label of the test case to be evaluated according to the maximum value of the grade label probability; Among them, the test case quality classification method is any one of the above-mentioned test case quality classification method embodiments.

[0041] Specifically, a large language model such as StarCoder is used to automatically generate multiple test cases to be evaluated for the code to be tested, and the multiple test cases to be evaluated are used as inputs of multiple quality classification models to obtain the test cases to be evaluated with a value higher than 1 to higher than The quality classification probability, in this embodiment N=4, that is, there are 3 quality classification models, which are respectively used to calculate the quality classification probability of the test case to be evaluated with a level label higher than 1, the quality classification probability of the level label higher than 2, and the quality classification probability of the level label higher than 3. Based on these three quality classification probabilities, according to the objective function of the level label probability, the level label probability is calculated in turn, and the level label of the test case to be evaluated is determined according to the maximum value of the level label probability.

[0042] This embodiment determines the quality classification probabilities of the test cases to be evaluated through the quality classification model; constructs the objective function of the grade label probability according to the quality classification probability, calculates the grade label probabilities of the test cases to be evaluated in turn according to the objective function and the quality classification probability, and selects the grade label corresponding to the maximum grade label probability as the grade label of the test case to be evaluated. This embodiment calculates the grade label probabilities according to the quality classification probability of the test case, and selects the scoring grade with the largest grade label probability as the grade label of the test case to be evaluated, thereby achieving accurate and multi-level scoring of the test cases, and providing an accurate basis for the application of the test cases.

[0043] In some embodiments of the present invention, an objective function of the grade label probability is constructed according to the quality classification probability, and the grade label probability objective function is: , in is the number of level labels, is the level label, and is a positive integer greater than 1, is the probability value of the level label, that is, the level label of the test case is The probability of is the quality classification probability, that is, the test case grade label is greater than probability.

[0044] In some embodiments of the present invention, Figure 5 As shown, Figure 5 The present invention provides Figure 4 The flowchart of an embodiment of step S402 in FIG. 4 is a flowchart of an embodiment of step S402, wherein the probability of each level label of the test case to be evaluated is calculated according to the objective function and the quality classification probability, and the level label of the test case to be evaluated is determined according to the maximum value of the level label probability, including: S501, calculating the probability of each level label of the test case to be evaluated according to the level label probability objective function and the quality classification probability; S502: According to the level label probability, select the level label corresponding to the maximum level label probability as the level label of the test case to be evaluated.

[0045] Specifically, all quality classification probabilities of the test cases to be evaluated are substituted into the corresponding objective functions, and the probability of each level label of the test cases to be evaluated is calculated in turn. For example, the probability of a quality label of test case t having a level label higher than 1 is calculated to be 0.9 through the quality classification probability model, that is, , the probability of a quality label with a level label higher than 2 is calculated by the quality classification probability model to be 0.7, that is , the probability of a quality label with a level label higher than 3 is calculated by the quality classification probability model to be 0.5, that is , calculate the probability of scoring 1, 2, 3 and 4 in turn, , visible, is the maximum level label probability, then the level label of the test case t is 4.

[0046] This embodiment calculates the grade label probabilities of the test cases to be evaluated in turn according to the quality classification probabilities of the test cases to be evaluated, determines the final scoring levels through the grade label probabilities, and implements accurate multi-level quality scoring of the test cases.

[0047] In order to better implement the test case scoring method in the embodiment of the present invention, the following examples are used to illustrate the application scenarios of the implementation of the present invention. It should be understood that the following application scenarios are only applicable to the application scenarios of the scheme of the present invention and do not constitute a limitation on the application scenarios of the scheme of the present invention.

[0048] Generate test cases for the code under test , , , as well as , test case Through the quality classification model and the grade label probability objective function, we get its grade label as 4, and the grade label probability of grade label 4 is , test case Through the quality classification model and the grade label probability objective function, we get its grade label as 4, and the grade label probability of grade label 4 is , test case Through the quality classification model and the grade label probability objective function, we get its grade label as 3, and the grade label probability of grade label 3 is , test case Through the quality classification model and the grade label probability objective function, we get its grade label as 3, and the grade label probability of grade label 3 is , test case Through the quality classification model and the grade label probability objective function, we get its grade label as 1, and the grade label probability of grade label 1 is , sort the five test cases by quality score from high to low, where and The level label is 4, but The probability of a class label is less than The higher the probability of the level label, the better the quality of the test case. The quality is better than The quality of The quality is better than The quality of these five test cases is ranked from high to low in terms of quality. , , , , ,Finally, the appropriate threshold can be determined according to the ,test cases to filter the test cases ,and the test cases that are higher than the preset threshold can be ,selected as the optimal test cases.

[0049] In order to better illustrate the advantages of the test case quality classification and scoring method in this embodiment, the optimal test cases obtained by the method of this embodiment are compared one by one with the test cases generated by three automated testing tools, Evosuite, AthenaTest and ChatGPT, in terms of grammatical correctness, compilation correctness, execution correctness, statement coverage and branch coverage, as shown in Table 1.

[0050]

[0051] Table 1 Comparison of test case data screened by the method of the present invention and generated by other methods Specifically, in terms of grammatical correctness, the grammatical correctness of the test cases screened by the method of this embodiment is 97.3%; in terms of compilation correctness, the compilation correctness of the test cases screened by the method of this embodiment is 73.4%; in terms of execution correctness, the execution correctness of the test cases screened by the method of this embodiment is 41.4%; in terms of statement coverage, the coverage of the test cases screened by the method of this embodiment is 70.3%; in terms of branch coverage, the branch coverage of the test cases screened by the method of this embodiment is 69.4%. In summary, the test cases screened by the method of this embodiment have good quality considering comprehensive grammatical correctness, compilation correctness, execution correctness, statement coverage and branch coverage, and can more efficiently provide high-quality test cases for testers, thereby improving software testing efficiency and reliability.

[0052] In order to better implement the test case quality classification method in the embodiment of the present invention, based on the test case quality classification method, correspondingly, Figure 6 As shown, the embodiment of the present invention further provides a test case quality classification device, and the test case quality classification device 600 includes: A grade label determination module 601 determines the grade of the test case and determines the grade label of the test case according to the grade; A quality label allocation rule generating module 602 is configured to set a quality label allocation rule according to the level label, wherein the number of the level label is one more than the number of the quality label allocation rule; A quality label training data set generation module 603, based on the grade labels, sequentially uses the quality label assignment rules to assign quality labels to the test cases to generate the same number of quality label training data sets as the quality label assignment rules; The quality classification module 604 constructs a corresponding number of quality classification models based on the quality labels according to the quality label training data set, determines the quality classification probability of the test case to be evaluated according to the quality classification model, and determines the quality classification of the test case to be evaluated according to the quality classification probability.

[0053] The test case quality classification device 600 provided in the above embodiment can implement the technical solution described in the above test case quality classification method embodiment. The specific implementation principles of the above modules or units can refer to the corresponding contents in the above test case quality classification method embodiment, which will not be repeated here.

[0054] In order to better implement the test case scoring method in the embodiment of the present invention, based on the test case scoring method, correspondingly, Figure 7 As shown, the embodiment of the present invention further provides a test case scoring device, and the test case scoring device 700 includes: The quality classification probability generating module 701 determines the quality classification probability of each test case to be evaluated based on the test case quality classification method; Scoring module 702 constructs an objective function of grade label probability according to the quality classification probability, calculates the grade label probability of each test case to be evaluated according to the objective function and the quality classification probability, and determines the grade label of the test case to be evaluated according to the maximum value of the grade label probability.

[0055] The test case scoring device 700 provided in the above embodiment can implement the technical solution described in the above test case scoring method embodiment. The specific implementation principles of the above modules or units can refer to the corresponding contents in the above test case scoring method embodiment, which will not be repeated here.

[0056] In the embodiment of the present invention, the test case quality classification device and scoring device can be an independent server, or a server network or server cluster composed of servers. For example, the test case quality classification device and scoring device described in the embodiment of the present invention include but are not limited to computers, network hosts, single network servers, multiple network server sets or cloud servers composed of multiple servers. The cloud server is composed of a large number of computers or network servers based on cloud computing.

[0057] The present invention also provides a test case quality classification and scoring electronic device, such as Figure 8 As shown, Figure 8 The structural block diagram of an embodiment of the test case quality classification and scoring device provided by the present invention. The test case quality classification and scoring device 800 can be a computing device such as a mobile terminal, a desktop computer, a notebook, a palm computer, and a server. The test case quality classification and scoring device 800 includes a processor 801 and a memory 802, wherein the memory 802 stores a test case quality classification and scoring program 803.

[0058] In some embodiments, the memory 802 may be an internal storage unit of a computer device, such as a hard disk or memory of the computer device. In other embodiments, the memory 802 may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the computer device. Further, the memory 802 may also include both an internal storage unit of the computer device and an external storage device. The memory 802 is used to store application software and various types of data installed in the computer device, such as program codes installed in the computer device. The memory 802 may also be used to temporarily store data that has been output or is to be output. In one embodiment, the test case quality classification and scoring program 803 may be executed by the processor 801, thereby realizing the test case quality classification and scoring method, device and electronic device of each embodiment of the present invention.

[0059] In some embodiments, the processor 801 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run program codes stored in the memory 802 or process data, such as executing test case quality classification and scoring procedures.

[0060] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0061] The test case quality classification and scoring method, device and electronic device provided by the present invention are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A test case quality classification method, characterized in that: include: Determine a rating level for the test case, and determine a level label for the test case according to the rating level; Setting a quality label allocation rule according to the grade label, wherein the number of the grade labels is one more than the number of the quality label allocation rules; Based on the grade labels, assigning quality labels to the test cases in turn according to the quality label assignment rules to generate a quality label training data set with the same number as the quality label assignment rules; A corresponding number of quality classification models based on the quality labels are constructed according to the quality label training data set, the quality classification probabilities of the test cases to be evaluated are determined according to the quality classification models, and the quality classifications of the test cases to be evaluated are determined according to the quality classification probabilities.

2. The test case quality classification method according to claim 1, characterized in that: Determining a rating level of the test case, and determining a level label of the test case according to the rating level, including: Conduct quality assessment of test cases based on the preset software test management tool and determine the test case scoring level; Map the test cases to the grade labels according to the rating levels to generate grade labels for the test cases.

3. The test case quality classification method according to claim 2, characterized in that: Setting a quality label assignment rule according to the grade label, and sequentially using the quality label assignment rule to assign quality labels to the test cases to generate a quality label training data set with the same number as the quality label assignment rule, including: Set the number of level labels to N; According to the N grade labels, set N-1 quality label allocation rules; According to the N-1 quality label assignment rules, test cases with labels greater than i in the test cases are assigned positive quality labels, and test cases with level labels less than or equal to i are assigned negative quality labels, where: , and are all positive integers greater than 1, generating A quality labeled training dataset.

4. The test case quality classification method according to claim 3, characterized in that: Constructing a corresponding number of quality classification models based on the quality labels according to the quality label training data set, determining the quality classification probability of the test case to be evaluated according to the quality classification model, and determining the quality classification of the test case to be evaluated according to the quality classification probability, including: According to the The CodeT5 model is iteratively trained on each quality label training dataset to determine the negative quality classification probability of the test cases in each training dataset. According to the negative quality classification probability, the positive quality classification probability is calculated, and according to the Quality label training dataset and its positive quality classification probability construction A quality classification model; According to the quality classification models, and determine the test cases to be evaluated in turn. Quality classification probability values; According to the The quality classification probability value determines the quality classification of the test case to be evaluated.

5. A test case scoring method, characterized in that: include: Determine the probability of each quality classification of the test case to be evaluated based on the test case quality classification method; Constructing an objective function of the grade label probability according to the quality classification probability, calculating the probability of each grade label of the test case to be evaluated according to the objective function and the quality classification probability, and determining the grade label of the test case to be evaluated according to the maximum value of the grade label probability; Wherein, the test case quality classification method is the test case quality classification method of any one of claims 1-4.

6. The test case scoring method according to claim 5, characterized in that: The objective function of the grade label probability is constructed according to the quality classification probability, and the grade label probability objective function is: ， in is the number of level labels, is the level label, and is a positive integer greater than 1, is the probability value of the level label, that is, the level label of the test case is The probability of is the quality classification probability, that is, the test case grade label is greater than probability.

7. The test case scoring method according to claim 6, characterized in that: Calculating the probability of each level label of the test case to be evaluated according to the objective function and the quality classification probability, and determining the level label of the test case to be evaluated according to the maximum value of the level label probability, including: According to the grade label probability objective function and the quality classification probability, the grade label probabilities of the test cases to be evaluated from 1 to N are calculated respectively; According to the level label probability, the level label corresponding to the maximum level label probability is selected as the level label of the test case to be evaluated.

8. A test case quality classification device, characterized in that: include: A grade label determination module determines the grade of the test case and determines the grade label of the test case according to the grade; A quality label allocation rule generation module is configured to set a quality label allocation rule according to the grade label, wherein the number of the grade label is one more than the number of the quality label allocation rule; A quality label training data set generation module, based on the grade labels, sequentially uses the quality label assignment rules to assign quality labels to the test cases to generate the same number of quality label training data sets as the quality label assignment rules; A quality classification module constructs a corresponding number of quality classification models based on the quality labels according to the quality label training data set, determines the quality classification probability of the test case to be evaluated according to the quality classification model, and determines the quality classification of the test case to be evaluated according to the quality classification probability.

9. A test case scoring device, characterized in that: include: A quality classification probability generation module determines the quality classification probabilities of the test cases to be evaluated based on the test case quality classification method; The scoring module constructs an objective function of the grade label probability according to the quality classification probability, calculates the probability of each grade label of the test case to be evaluated according to the objective function and the quality classification probability, and determines the grade label of the test case to be evaluated according to the maximum value of the grade label probability.

10. An electronic device, characterized in that: It includes a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the test case quality classification method as described in any one of claims 1 to 4 are implemented, and / or the steps of the test case scoring method as described in any one of claims 5 to 7 are implemented.

Images