A software regression test case priority ordering method considering defect distribution

Abstract

The application belongs to the technical field of software testing, and specifically discloses a software regression test case priority sequencing method considering defect distribution, which comprises the following steps: obtaining a historical defect data set of software, dividing the historical defect data set into a training set and a verification set, inputting a machine learning classification algorithm, constructing a model, and obtaining performance evaluation results; extracting a current version metric element of the software under test to input the model to obtain a defect prediction probability; extracting dynamic code coverage information of the test case of the software under test, weighting and integrating the defect prediction probability and the metric element, and constructing a linear function as a priority sequencing objective function; sequencing through a sequencing strategy to obtain a test case priority sequencing result; and the method has the following advantages: considering defect distribution, metric elements, code coverage and other factors for linear weighting, solving the problem that existing methods do not consider defect distribution and reduce defect detection rate, or rely on prior knowledge when multiple factors are weighted, leading to unstable sequencing results, and improving the performance of the software test case priority model.

Description

Technical Field

[0001] This invention relates to the field of software testing technology, and more specifically, to a method for prioritizing software regression test cases that takes into account defect distribution. Background Technology

[0002] Software testing, as a crucial means of verifying software quality, is frequently used in activities aimed at ensuring software quality. However, when software testing resources are limited, such as budget, time, and manpower, treating all test cases as having the same contribution to the testing objective can prevent the prioritization of test cases closer to the target, such as those more likely to expose faults or provide greater code coverage. This hinders developers from efficiently proceeding to the next steps, such as software debugging and fault fixing, impacting the software development cycle. Test Case Prioritization (TCP) helps software testers achieve testing objectives more quickly, such as completing code coverage or discovering defects earlier, allowing developers to begin debugging sooner and saving time and manpower costs.

[0003] The most commonly used traditional TCP technologies based on dynamic code coverage are Total and Additional. However, traditional technologies only utilize the dynamic code coverage information of test cases, and the execution result information is not fully utilized. The premise of Total and Additional is that "the higher the code coverage, the greater the likelihood that the test cases will reveal defects," which implicitly assumes that "defects are uniformly distributed in the software." Clearly, the "80 / 20 rule" of defect distribution in software makes considering only dynamic code coverage flawed. Considering the distribution of defects can break the premise of only considering dynamic code coverage, i.e., breaking the assumption that defects are uniformly distributed in code entities. Existing TCP technologies based on defect prediction and dynamic code coverage combine defect prediction information and dynamic code coverage information to improve the defect detection rate in software testing. Integrating the defect prediction method (Total) and the dynamic code coverage method (Additional) to prioritize test cases presents challenges. Setting the fusion coefficients for multiple factors requires prior knowledge of software defects, such as the number of defects in the software. Furthermore, prior knowledge often involves fixed coefficients that are difficult to obtain, and the integration factors do not use metrics. This results in different parameter settings having different effects on the priority ranking of software test cases, and the influence of metrics on the priority of software test cases.

[0004] To address this issue, a software regression test case priority ranking method that considers defect distribution is proposed. Summary of the Invention

[0005] The present invention aims to provide a software regression test case priority ranking method that considers defect distribution, in order to solve or improve the problem in the above-mentioned technical problem that when combining defect prediction and test case priority ranking methods, the fusion coefficient setting depends on prior knowledge, which leads to a decrease in the test defect detection rate of the ranking result.

[0006] In view of this, a first aspect of the present invention is to provide a method for prioritizing software regression test cases that takes into account defect distribution.

[0007] The first aspect of the present invention provides a method for prioritizing software regression test cases considering defect distribution, comprising the following steps: S1, obtaining a historical defect dataset of software with the same metrics as the software under test and converting it into a training set, inputting it into a machine learning classification algorithm for training, and constructing a defect tendency prediction model based on the trained input machine learning classification algorithm; S2, obtaining a historical defect dataset of a certain software under test as a validation set, and inputting it into the defect tendency prediction model to obtain the performance evaluation result of the defect tendency prediction model; S3, extracting the metrics of the software under test in the current version as a test set, and inputting it into the defect tendency prediction model to obtain the defect prediction probability of each entity of the software under test; S4, extracting the dynamic code coverage data of the test cases of the software under test, and performing a weighted integration with the defect prediction probability, metrics, and performance evaluation result to obtain a test case metric improvement linear function; S5, using the test case metric improvement linear function as the priority ranking objective function, and ranking the priority of all test cases according to the ranking strategy to obtain the priority ranking result.

[0008] Furthermore, the historical defect dataset is defined as D, and the historical defect dataset D includes: the metrics of the software under test. And the defect labeling situation L={l1,l2,…,l m} T The steps for obtaining it are as follows: determining whether the amount of historical version defect data for the software under test is sufficient; when the historical version data for the software under test is sufficient, obtaining the historical related defect dataset D. W The historical defect dataset D is used to construct the validation set D, and a specific historical version of the data is selected from D. V The remaining data in D is used as the training set D. Tr Alternatively, when historical version data for the software under test is insufficient, obtain a historical dataset of relevant defects for the software under test, D. Wi And the historical related defect dataset D of other software with the same metrics as the software under test C Used to construct the historical defect dataset D, in D C Select a specific historical version of data as the validation set D VCombine the remaining data in D with D Wi As training set D Tr Where A represents an m×n metric matrix obtained by measuring m entities in the software using n metrics, m represents the number of entities in dataset D, and n represents the number of metrics in the dataset. l represents the nth metric value of the mth entity. m Let L represent the defect label value of the m-th entity, and L represent the defect label information for all entities.

[0009] Furthermore, the training set D Tr The formula is as follows:

[0010]

[0011] In the formula, For the k-th sample, the n-th metric value. The validation set D represents the defect label value of the k-th sample. V The formula is as follows:

[0012]

[0013] In the formula, For the z-th sample and the n-th metric value, l Vz This represents the defect label value of the z-th sample; and step S2 specifically includes: S201, processing the training set D Tr Input the machine learning classification model for training to construct a defect tendency prediction model; S202, input the validation set D V metric A V Input the defect propensity prediction model to obtain the defect prediction probability. Where L Vp Let L be a z×1 vector; S203, let L be the defect prediction probability. Vp With L V Input the performance evaluation metric formula F of the machine learning classification model to obtain the performance evaluation result value Per of the Model, and calculate it using the following formula: Per = F(L) Vp ,L V ).

[0014] Further, step S3 specifically includes: S301, extracting the metrics of the tested software in the current version to obtain the current version's metric A. cur S302, change the current version of the metric A cur This serves as a test set, which is then input into the defect propensity prediction model to obtain the defect prediction probability L for each entity of the software under test. curp Among them, Acur Let q×n be a matrix consisting of the metric values ​​of q entities in the software under test, where each entity has n metrics, and specifically expressed as follows:

[0015] Furthermore, step S4 specifically includes: using an improved linear weighting method to calculate the defect prediction probability L. curp A cur A certain metric A in j Integrating with dynamic code coverage information Cover, and using the performance evaluation result Per as the weighting coefficient of dynamic code coverage information, a linear function f for test case metric improvement is obtained, specifically as follows: Where A is A j The result of normalization, Cover is dynamic code coverage information, A min For A j Minimum value, A max For A j The maximum value.

[0016] Furthermore, the sorting strategy is Additional or Total.

[0017] The beneficial effects of this invention compared to the prior art are as follows:

[0018] For software test case prioritization, this paper combines defect prediction and test case prioritization methods, simultaneously considering multiple factors including metrics, defect prediction probabilities, and dynamic code coverage information. The performance evaluation results of the defect prediction model are used as weighting coefficients for the sum of metrics, defect prediction probabilities, and dynamic code coverage information, thus addressing the issue of weighting coefficients relying on prior knowledge when integrating multiple factors. By considering defect distribution (i.e., metrics and defect prediction probabilities) and code coverage, the performance of the software test case prioritization model and defect detection rate are improved, guiding software testing strategies, thereby shortening software development cycles and saving costs.

[0019] Additional aspects and advantages of embodiments of the invention will become apparent in the following description or may be learned by practice of embodiments of the invention. Attached Figure Description

[0020] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0021] Figure 1 This is a flowchart of the software regression test case priority ranking method considering defect distribution according to the present invention.

[0022] Figure 2Statistical data of open-source Java projects provided for embodiments of the present invention;

[0023] Figure 3 The usage of datasets for the two scenarios of CVDP and CPDP-CM provided for embodiments of the present invention;

[0024] Figure 4 The parameter settings of the comparison method provided for embodiments of the present invention;

[0025] Figure 5 The CVDP scenario provided in the embodiments of the present invention and the comparison method Modified yield the mean APFD results of the Additional and Total strategies.

[0026] Figure 6 The CVDP scenario provided in the embodiments of the present invention compares the ΔAPFD results obtained by the present invention and the comparative method Modified under the Additional and Total strategies;

[0027] Figure 7 The APFD mean results obtained by the present invention and the comparative method QTEP in the CPDP-CM scenario provided in the embodiments of the present invention are based on the Additional and Total strategies.

[0028] Figure 8 The CPDP-CM scenario provided in the embodiments of the present invention is compared with the QTEP method in terms of the △APFD results obtained by the present invention and the additional and total strategies.

[0029] CVDP stands for Cross Version Defect Prediction, CPDP-CM stands for Homogeneous Defect Prediction, also known as CrossProject Defect Prediction with Common Metrics, and APFD refers to the TCP performance evaluation metric: Weighted Average of the Percentage of Faults Detected. Detailed Implementation

[0030] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0031] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0032] Please see Figure 1-8 The following describes a method for prioritizing software regression test cases that considers defect distribution, based on some embodiments of the present invention.

[0033] An embodiment of the first aspect of the present invention proposes a method for prioritizing software regression test cases that considers defect distribution. In some embodiments of the present invention, such as... Figure 1 As shown, a method for prioritizing software regression test cases that considers defect distribution is provided. This method includes:

[0034] S1. Obtain historical defect datasets for software with the same metrics as the software under test and convert them into training sets. Input the datasets into machine learning classification algorithms for training. Construct a defect tendency prediction model based on the trained input machine learning classification algorithms.

[0035] S2, obtain a historical defect dataset for a certain software under test as a validation set, and input it into the defect tendency prediction model to obtain the performance evaluation results of the defect tendency prediction model;

[0036] S3, extract the metrics of the software under test in the current version as the test set, and input them into the defect tendency prediction model to obtain the defect prediction probability of each entity of the software under test.

[0037] S4. Extract the dynamic code coverage data of the test cases of the software under test, and perform weighted integration with the defect prediction probability, metrics and performance evaluation results to obtain a linear function for test case metric improvement.

[0038] S5 uses the linear function for improving test case metrics as the priority ranking objective function.

[0039] Based on the sorting strategy, the priority of all test cases is sorted to obtain the priority ranking result.

[0040] This invention provides a software regression test case priority ranking method that considers defect distribution. For the task of prioritizing software test cases, it combines defect prediction and test case priority ranking methods, and automatically assigns multiple factors such as integrated metrics, defect prediction probability and dynamic code coverage information. This solves the problem that the setting of fusion coefficients depends on prior knowledge, improves the performance of the software test case priority model and the defect detection rate, enables guidance for software testing strategies, and thus shortens the software development cycle and saves costs.

[0041] In any of the above embodiments, the historical defect dataset is set as D, and the historical defect dataset D includes: the metrics of the software under test. And the defect labeling situation L={l1,l2,…,l m} T The steps for obtaining it are as follows:

[0042] Determine whether the amount of historical defect data for the software under test is sufficient;

[0043] When sufficient historical version data of the software under test is available, obtain the historical relevant defect dataset D. W The historical defect dataset D is used to construct the validation set D, and a specific historical version of the data is selected from D. V The remaining data in D is used as the training set D. Tr ;or

[0044] When historical version data of the software under test is insufficient, obtain the historical relevant defect dataset D of the software under test. Wi And the historical related defect dataset D of other software with the same metrics as the software under test C Used to construct the historical defect dataset D, in D C Select a specific historical version of data as the validation set D V Combine the remaining data in D with D Wi As training set D Tr ;

[0045] Where A represents an m×n metric matrix obtained by measuring m entities in the software using n metrics, m represents the number of entities in dataset D, and n represents the number of metrics in the dataset. l represents the nth metric value of the mth entity. m Let L represent the defect label value of the m-th entity, and L represent the defect label information for all entities.

[0046] In this embodiment, there is sufficient historical data, and the historical data and the software under test come from the same software, resulting in a better performance of the constructed defect prediction model;

[0047] Insufficient data, specifically a lack of training data, makes it impossible to train a model. Using historical data from other software can build a defect prediction model, although its performance is generally not as good as a model built with sufficient historical data, but the model is usable and solves the problem of insufficient data.

[0048] Specifically, the threshold for determining whether a sample is sufficient is a constant, which is set to 100 samples in this invention.

[0049] In any of the above embodiments, the training set D Tr The sum is given by the following formula:

[0050]

[0051] In the formula, A Tr Let l be a k×n matrix representing the metric values ​​of the training set. Tr Let be the defect label values ​​of the training set, and be a k×1 vector. For the k-th sample, the n-th metric value. This represents the defect label value of the k-th sample;

[0052] Validation set D V The formula is as follows:

[0053]

[0054] In the formula, A V A z×n matrix representing the metric values ​​of the validation set; V Let be the defect label values ​​of the validation set, and be a z×1 vector. For the z-th sample and the n-th metric value, l Vz This represents the defect label value of the z-th sample;

[0055] And step S2 specifically includes:

[0056] S201, training set D Tr Input the machine learning classification model for training and build a defect tendency prediction model.

[0057] S202, verify set D V metric A V Input the defect propensity prediction model to obtain the defect prediction probability. Where L Vp A vector of size z × 1;

[0058] S203, the defect prediction probability L Vp With L V Input the performance evaluation metric formula f for the machine learning classification model to obtain the performance evaluation result value Per of the Model, and calculate it using the following formula:

[0059] Per = F(L) Vp ,L V ).

[0060] In any of the above embodiments, step S3 specifically includes:

[0061] S301, Extract the metrics of the software under test in the current version to obtain the current version's metric A. cur ;

[0062] S302, the current version of metric A cur This serves as the test set, which is then input into the defect propensity prediction model to obtain the defect prediction probability L for each entity in the software under test. curp ;

[0063] Among them, A cur Let q×n be a matrix consisting of the metric values ​​of q entities in the software under test, where each entity has n metrics, and specifically expressed as follows:

[0064]

[0065] In this embodiment, defect prediction probability and code coverage are two completely different dimensions. Typically, the former has a coarser granularity, such as at the class level, while the latter has a finer granularity, such as at the unit level. Therefore, based on the correlation between the two, this invention maps the parameters of the two different dimensions to the dimension of the finer granular parameter, namely the dimension of code coverage.

[0066] In any of the above embodiments, step S4 specifically includes:

[0067] An improved linear weighting method is used to calculate the defect prediction probability L. curp A cur A certain metric A in j Integrating with dynamic code coverage information Cover, and using the performance evaluation result Per as the weighting coefficient of dynamic code coverage information, a linear function f for test case metric improvement is obtained, specifically as follows:

[0068]

[0069] Where A is A j The result of normalization, Cover is dynamic code coverage information, A min For A j Minimum value, A max For A j The maximum value.

[0070] In this embodiment, the obtained evaluation result Per is set to w1, i.e., w1 = Per. Because this invention only considers the impact of three factors—defect prediction probability, metrics, and code coverage—on test case priority ranking, and the defect prediction probability and metrics are linearly averaged into a single set, w2 = 1 - w1 is set. This yields the linear function for improving test case metrics.

[0071] The reliability of defect prediction probability can be understood as follows: its function is to "assign weights to code units to avoid ignoring code units that are predicted to have defective tendencies, so that the test case priority ranking objective function uses defect prediction probability as one of the bases". The reliability of code coverage w2 can be understood as "assigning weights to code units to avoid ignoring code coverage of code units that are predicted to have no defective tendencies, so that the test case priority ranking objective function uses code coverage as another base".

[0072] In any of the above embodiments, the sorting strategy is Additional or Total.

[0073] Another embodiment of the first aspect of the present invention proposes a method for prioritizing software regression test cases that considers defect distribution. In some embodiments of the present invention, such as... Figure 1-8 As shown, a method for prioritizing software regression test cases that considers defect distribution is provided. This method includes:

[0074] S1. Extract the historical defect dataset of the software as a training set, input it into a machine learning classification algorithm, and build a defect tendency prediction model;

[0075] S2. Extract a defect dataset from a historical version of the software under test as a validation set, input it into the defect tendency prediction model, and obtain the performance evaluation results of the defect tendency prediction model;

[0076] S3. Extract the metrics of the software under test as the test set, and input them into the defect tendency prediction model to obtain the defect prediction probability of the software entity under test.

[0077] S4. Extract dynamic code coverage information of test cases for the software under test;

[0078] S5. Combine the three factors of defect prediction results, metrics, and dynamic code coverage information of test cases, and construct a test case metric improvement linear function using the evaluation results of the defect propensity prediction model;

[0079] S6. Based on the test case metrics, improve the linear function and sorting strategy to obtain the priority ranking results of software regression testing.

[0080] In some embodiments, the specific content of S1 includes:

[0081] S11. Historical defect dataset D includes software static metrics. and the corresponding defect status label L={l1,l2,…,l m} TWhere m represents the number of samples in the dataset, and n represents the number of static metrics. When sufficient historical version data of the software under test is available, only the relevant defect dataset {D} of the historical versions of the software under test is collected. W1 D W2 D Wk} as historical defect dataset D W D W =D; When historical version data of the software under test is insufficient, collect historical version data D of the software under test. Wi Related defect datasets for historical versions of other project software {D C1 D C2 D Ck} as historical defect dataset D C D C =D;

[0082] S12. In the collected historical defect dataset D, remove a specific historical version of the software under test from the dataset D. Wi The remaining historical defect dataset is used as the training set for the defect propensity prediction model, and input into a machine learning classification algorithm to construct the defect propensity prediction model DPM; that is, when the collected historical defect dataset is D... W When, select {D W1 D W2 D Wi-1 D Wi+1 D Wk The historical defect dataset D is used as the training set for the defect propensity prediction model. C At that time, remove data from a specific historical version of the software under test, D. Wi The remaining historical defect dataset {D C1 D C2 D Cn The set is used as the training set for the defect propensity prediction model, and then input into the machine learning classification algorithm to construct the defect propensity prediction model DPM.

[0083] In some embodiments, the specific content of S2 includes:

[0084] S21. Take the remaining defect datasets from the historical defect dataset D collected in S11. As a verification set, D W1 Static metric metadata in Input DPM, get A Wi The corresponding defect prediction probability result L Vp ={l Wp1 ,l Wp2 ,…,l Wpm}T ;

[0085] S22. Move L from S21 Vp and A Wi Corresponding real label L Wi ={l W1 ,l W2 ,...,L Wm} T The comparison was performed using a performance evaluation metric with a classification model value range of [0, 1] to obtain the evaluation result Per;

[0086] In this embodiment, the defect data extracted from the second-to-last version of the software under test is used as the validation set of the prediction model. For example, if the current version of the software under test is V1 and the previous version is V2, then V2 is used as the version of the validation set.

[0087] In this embodiment, the evaluation index Balance of the binary classification algorithm is used as the evaluation index Per of the defect tendency prediction model, and its calculation method is shown in the following formula:

[0088]

[0089] In this context, PD refers to the Probability of Detection, PF refers to the Probability of False Alarm (PF), and TP, FP, TN, and FN are the confusion matrices used in evaluating the performance of the classification model: true positives (TP), false positives (FP), true negatives (TN), and false negatives (TP). TP indicates that the true class label of a positive sample matches the predicted class label; TN indicates that the true class label of a negative sample matches the predicted class label; FP indicates that the class label of a negative sample is predicted as a positive sample; and FN indicates that the class label of a positive sample is predicted as a negative sample. The reason for choosing Balance is that this index was proposed by Menzies et al. (Menzies T., Greenwald J., Frank A. Data Mining Static Code Attributes to Learn Defect Predictors[J].IEEE Transactions on Software Engineering, 2007, 33(1):2-13) for evaluating the performance of defect tendency prediction models under unbalanced data conditions. It is a comprehensive index used in many defect tendency prediction models, and different defect tendency prediction models will produce different Balance values ​​for the target software. In addition, this coefficient will not have a null value "NAN" and its value range is [0, 1].

[0090] In some embodiments, the specific content of S3 includes:

[0091] S31. Extract the static metrics A of the current version of the software under test. cur ;

[0092] In this embodiment, static metric A is selected. cur The number of lines of code (LoC) of software entities in the dataset is used as a static metric A. j ;

[0093] S32. Place A cur The test set is input into the S21 defect tendency prediction model DPM, and the defect prediction probability L of each entity in the software under test is obtained. curp ;

[0094] In this embodiment, the defect data extracted from the latest version of the software under test is used as the test set for the prediction model. For example, if the current version of the software under test is V1, then V1 is used as the version of the test set.

[0095] In some embodiments, the specific content of S5 includes:

[0096] S51. The defect prediction results L of the current version of the software under test in S32. curp A static metric A extracted from S31 j Linear integration of the dynamic code coverage information Cover extracted from S4 for test cases yields the following formula;

[0097]

[0098] Among them, A max A represents j The maximum value, A min A represents j The minimum value, w1 represents L curp and A j The weighting coefficients are w2 and w2 represents the weighting coefficient of Cover. It's important to note that defect prediction probability and code coverage are two completely different dimensions. Typically, the former has a coarser granularity (e.g., class level), while the latter has a finer granularity (e.g., unit level). Therefore, this invention maps the parameters of these two different dimensions to the finer-grained parameter dimension, namely the code coverage dimension, based on their correlation.

[0099] S52. Set the evaluation result Per obtained in S21 to w1, that is, w1 = Per, w2 = 1 - w1. The linear function for improving test case metrics can then be obtained, as shown in the following equation:

[0100]

[0101] In some embodiments, the specific content of S6 includes:

[0102] S6. Based on the test case metrics, improve the linear function as the priority ranking objective function, and obtain the priority ranking results of software regression tests according to the ranking strategy.

[0103] S61. Based on S5, the linear function of test case measurement is improved as the priority objective function. Combined with the ranking strategy Additional or Total, the priority of software regression test cases is ranked to obtain the priority ranking result of software regression test cases.

[0104] S62. Based on the sorting results in S61, the performance of the sorting model is evaluated using the performance metrics of the test case priority sorting model.

[0105] In this embodiment, as Figure 2The defect database from the publicly available Java project Defects4J+M by Hosseinabadi's team at Shareef University of Technology (Mostafa Mahdieh, Seyed-Hassan Mirian-Hosseinabadi, Khashayar Etemadi, et al. Incorporating fault-proneness estimations into coverage-based test case prioritization methods[J]. Information and Software Technology, 2020, 121) was used to construct the defect prediction model and prioritize software regression test cases. This database contains five categories: Chart, Closure, Lang, Math, and Time. Chart has 26 versions, Closure has 133 versions, Lang has 65 versions, Math has 106 versions, and Time has 27 versions. Each version contains metrics and defect information (i.e., defect data), test coverage information, failed test case information, and fault repair information. This section uses 75 datasets—versions 1-15 of graphs, languages, mathematics, and time, and version 1-15 of closures—as target projects for defect prediction and test case prioritization. The defect datasets are class-level granularity, with each sample representing a software module. When constructing the defect prediction model, the metrics in the samples are independent variables, and the defect information is the dependent variable. Numerical defect information is converted to a binary type: a defect count of 0 in the last column of the dataset is set to 0 (no defect), and a defect count greater than or equal to 1 is set to 1 (defect present). The defect rate of the defect dataset is the ratio of the number of defective samples to the total number of samples. The test coverage information is unit-level granularity; a defect dataset typically involves multiple unit-level test coverage information. Therefore, when prioritizing test cases, the entity granularity of the test case coverage and the entity granularity of the defect prediction metrics need to be converted to the same dimension before being sorted according to the formula of the ranking objective function.

[0106] In this embodiment, to compare with the method proposed in this invention (referred to as DDTCP), two software regression test case prioritization methods based on defect prediction were selected as comparison methods: the Modified Prioritization Method and the Quality Awareness Method (QTEP). Since the defect prediction models of the two comparison methods employ two different defect prediction scenarios—cross-version defect prediction and isomorphic defect prediction—to ensure fairness in the comparison, this invention uses the same scenarios as the comparison methods. The dataset usage for the corresponding scenarios is as follows: Figure 3 As shown, a defect prediction model is constructed using the same or recommended parameters described in the text, and the defect prediction and test coverage information are weighted accordingly. Figure 4 As shown, the experiment was conducted using the data processing software Python, and the experimental results are expressed as the mean.

[0107] In this embodiment, the weighted average percentage of faults detected (APFD), the most commonly used evaluation metric in TCP technology, is used to measure the defect discovery rate. APFD was proposed by Rothermel et al. in 1999 (Gregg Rothermel, Roland H Untch, Chengyun Chu, et al. Test Case Prioritization: An Empirical Study [A]. Proceedings of the International Conference on Software Maintenance (ICSM'99) [C]. Piscataway: IEEE, 1999: 179-188) to evaluate the rate at which software test cases detect faults. This metric assumes that all test cases have the same testing cost and that the fault level is consistent. Its definition is as follows:

[0108]

[0109] Among them, TF i Let APFD be the position in the sorting strategy T' where the first test case to find defect i is located, denoted as [0, 100%]. For the order in which test cases that fail are executed earlier, TF1 to TF2 are... m The smaller the value, the larger the APFD value. In other words, a higher APFD value indicates faster fault detection and better test performance for the corresponding test case ranking set. Random APFD ≈ 50%.

[0110] In this embodiment, the APFD relative improvement ratio (ΔAPDF) is the average improvement ratio of the APFD obtained by the method of the present invention to the baseline comparison method obtained on multiple versions. A positive ΔAPDF indicates that the method of the present invention is superior to the comparison method, and its calculation method is shown in the following formula:

[0111]

[0112] The software regression test case prioritization method proposed in this invention, which considers a multi-factor linear integration strategy, was used to prioritize and compare test cases across 75 versions in five categories: graph, closure, language, mathematical, and time. Representative results from Modified and QTEP methods are shown below. Figures 5 to 8 As shown, the percentage symbol is omitted from all data in the figure;

[0113] from Figures 5-6 The analysis shows that: (1) When using the Total strategy in the CVDP scenario, the DDTCP proposed in this invention achieves better results on most datasets compared with Modified (i.e., the priority ranking method based on modified total coverage); (2) When using the Additional strategy in the CVDP scenario, the DDTCP proposed in this invention achieves better results on some datasets compared with Modified (i.e., the priority ranking method based on modified additional coverage).

[0114] from Figures 7-8 The results and analysis show that when using the Total and Additional strategies in the CPDP-CM scenario, the DDTCP proposed in this invention achieves better results than the quality-aware method QTEP in most aspects, especially in terms of the relative improvement ratio △APFD across multiple versions.

[0115] Therefore, in summary, the DDTCP proposed in this invention has significant advantages over existing defect prediction-based software regression test case prioritization methods.

[0116] In the description of this invention, it should be understood that the terms "longitudinal", "lateral", "up", "down", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this invention, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention.

[0117] The above embodiments are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for prioritizing software regression test cases considering defect distribution, characterized in that, Includes the following steps: S1. Obtain historical defect datasets for software with the same metrics as the software under test and convert them into training sets. Input the datasets into machine learning classification algorithms for training and build defect tendency prediction models based on the trained machine learning classification algorithms. S2, Obtain a historical defect dataset for a certain software under test as a validation set, and input it into the defect tendency prediction model to obtain the performance evaluation results of the defect tendency prediction model; S3, extract the metrics of the software under test in the current version as the test set, and input them into the defect tendency prediction model to obtain the defect prediction probability of each entity of the software under test. S4. Extract the dynamic code coverage data of the test cases of the software under test, and perform weighted integration with the defect prediction probability, metrics and performance evaluation results to obtain a linear function for test case metric improvement. S5, using the test case metric improvement linear function as the objective function for priority ranking, and ranking the priorities of all test cases according to the ranking strategy to obtain the priority ranking result; The historical defect dataset is set as follows: D And historical defect dataset D Includes: the metric matrix of the software under test and defect labeling status The steps for obtaining it are as follows: Determine whether the amount of historical defect data for the software under test is sufficient; When sufficient historical version data of the software under test is available, acquire a historical dataset of relevant defects. D W Used to construct a historical defect dataset D ,exist D Select a specific historical version of data as the validation set. D V ,and D The remaining data is used as the training set. D Tr ;or When historical version data of the software under test is insufficient, obtain a historical dataset of relevant defects of the software under test. D Wi Historical related defect datasets of other software with the same metrics as the software under test. D C Used to construct a historical defect dataset D ,exist D C Select a specific historical version of data as the validation set. D V ,Will D C Remaining data and D Wi As a training set D Tr ; in, Indicates the software m Each entity n The dimensions obtained from the measurement of each indicator are: m × n Metric matrix, m Represents the dataset D The number of entities in n Indicates the number of metrics in the dataset. Indicates the first m The first entity n Each metric value, Indicates the first m The defect label value of each entity. L This indicates the defect label information for all entities. The training set D Tr The formula is as follows: ； In the formula, For the first k The first sample n Each metric value, Indicates the first k The defect label value of each sample; The verification set D V The formula is as follows: ； In the formula, For the first z The first sample n Each metric value, Indicates the first z The defect label value of each sample; And step S2 specifically includes: S201, training set D Tr Input the data into a machine learning classification model for training, and build a defect tendency prediction model. Model ; S202, verification set D V metric Input defect tendency prediction model Model To obtain the defect prediction probability ,in for z A vector of size ×1; S203, defect prediction probability and Input the performance evaluation metric formula for the machine learning classification model F to obtain Model Performance evaluation results And calculated using the following formula: = F ( , )； Step S3 specifically includes: S301, Extract the metrics of the tested software in the current version to obtain the metrics of the current version. A cur ; S302, the current version of the metric. A cur This serves as the test set, and the test set is input into the defect tendency prediction model. Model In order to obtain the defect prediction probability of each entity of the software under test. ; in, Indicates the software under test q The metric values ​​of each entity constitute q×n Matrix, and each entity has n Each metric element, specifically expressed as follows: ； Step S4 specifically includes: Using an improved linear weighting method , A cur A certain metric in A j and dynamic code overlay information Cover Perform integration and use performance evaluation results. As a weighting factor for dynamic code coverage information, a linear function for improving test case metrics is obtained. f Specifically, it is the following formula: ； in, for The result of normalization For dynamic code overlay information, for minimum value for The maximum value.

2. The method for prioritizing software regression test cases considering defect distribution according to claim 1, characterized in that, The sorting strategy is Additional or Total.

Images