Test case priority ranking method and device, electronic equipment and readable storage medium

Abstract

The invention discloses a test case priority ranking method and device, electronic equipment and a readable storage medium, and relates to the field of computer software testing. The method comprises the following steps: constructing a transaction module network of a target software system; extracting an evaluation feature of each test case in the plurality of test cases to be sorted, and inputting the evaluation features of the plurality of test cases into a priority evaluation model which completes training in advance to obtain a priority prediction value of each test case generated by the priority evaluation model; wherein the evaluation features of the test case comprise centrality features and network topology features of a transaction module corresponding to the test case in a transaction module network, and attribute features of the test case; and sorting the plurality of test cases according to the priority prediction values. Based on the technical scheme disclosed by the invention, the determined execution sequence of the test cases better meets the test requirements, so that the test efficiency is improved, and the test cost is reduced.

Description

Technical Field

[0001] The present application belongs to the field of computer software testing technology, and in particular, relates to a test case priority sorting method, device, electronic device and readable storage medium. Background Art

[0002] With the continuous development of the software industry, software testing has become increasingly indispensable in the software development life cycle. The importance of software testing lies in its ability to effectively discover and fix errors, vulnerabilities, and functional defects in software. As an effective means of ensuring software quality and reliability, software testing is crucial to ensuring that software products perform well in actual applications. At all stages of the software development life cycle, software testing consumes a lot of resources and funds, especially for industries with large systems such as the banking industry, the complexity and workload of software testing are even greater.

[0003] Software regression testing means re-running a series of test cases after the software is modified or updated to verify whether the modification or update of the software has introduced new errors or caused abnormalities in existing functions. In software regression testing, sorting test cases can help testers find defects in the software as early as possible, improve testing efficiency, shorten testing time, and thus reduce the cost of software testing.

[0004] For those skilled in the art, how to sort the test cases so that the execution order of the determined test cases better meets the test requirements, thereby improving the test efficiency and reducing the test cost is a problem that needs to be solved urgently. Summary of the invention

[0005] In view of the above problems, the present application provides a test case priority sorting method, device, electronic device and readable storage medium, so that the execution order of the determined test cases is more in line with the test requirements, thereby improving test efficiency and reducing test costs.

[0006] To achieve the above objectives, this application provides the following technical solutions:

[0007] The first aspect of the present application provides a test case priority sorting method, comprising:

[0008] Constructing a transaction module network of the target software system, wherein the transaction module network is a directed unweighted graph, the nodes in the directed unweighted graph are transaction modules in the target software system, and the relationship between the nodes is determined by the calling relationship between the transaction modules;

[0009] Extracting evaluation features of each test case from the plurality of test cases to be sorted; wherein the evaluation features of the test case include: centrality features and network topology features of the transaction module corresponding to the test case in the transaction module network, and attribute features of the test case;

[0010] Inputting the evaluation features of the plurality of test cases into a pre-trained priority evaluation model to obtain a priority prediction value of each test case generated by the priority evaluation model;

[0011] The multiple test cases are sorted according to the priority prediction values.

[0012] In a possible implementation, the step of constructing a transaction module network of a target software system includes:

[0013] Performing static analysis on the source code of the target software system to obtain the calling relationship between the transaction modules in the target software system;

[0014] A directed unweighted graph is constructed based on the calling relationship between the transaction modules in the target software system, and the directed unweighted graph is a transaction module network of the target software system.

[0015] In a possible implementation, the centrality characteristics of the transaction module corresponding to the test case in the target transaction module network include multiple or all of the following characteristics: degree, clustering coefficient, betweenness, closeness centrality, eigenvector centrality and shortest path length.

[0016] In one possible implementation, the training process of the priority evaluation model includes:

[0017] A training set is obtained, wherein the training set includes a plurality of training samples, each training sample includes an evaluation feature of a sample test case, and each training sample is annotated with a priority evaluation value; the evaluation feature of each sample test case includes: a centrality feature and a network topology feature of a transaction module corresponding to the sample test case in a transaction module network of a sample software system, and an attribute feature of the sample test case;

[0018] The model to be trained is iteratively trained based on the multiple training samples to optimize the model parameters of the model to be trained.

[0019] In a possible implementation, the training process of the priority evaluation model further includes:

[0020] Obtaining a validation set, the validation set comprising a plurality of validation samples, each validation sample comprising an evaluation feature of a sample test case, and each validation sample being annotated with a priority evaluation value;

[0021] The multiple verification samples are input into the model to be trained, the model to be trained generates a priority prediction value for each verification sample, and hyperparameters are adjusted based on the priority evaluation values ​​and priority prediction values ​​of the multiple verification samples.

[0022] In a possible implementation, the training process of the priority evaluation model further includes:

[0023] Screen the evaluation features used by the model to be trained.

[0024] In a possible implementation, the priority evaluation value of any sample test case is determined as follows:

[0025] Obtaining an execution frequency assessment value, an impact scope assessment value, an urgency assessment value, and a defect frequency assessment value of the sample test case;

[0026] The execution frequency evaluation value, impact scope evaluation value, urgency evaluation value and defect frequency evaluation value of the sample test case are weightedly summed to obtain the priority evaluation value of the sample test case.

[0027] A second aspect of the present application provides a test case priority sorting device, comprising:

[0028] A network construction module, used to construct a transaction module network of a target software system, wherein the transaction module network is a directed unweighted graph, the nodes in the directed unweighted graph are transaction modules in the target software system, and the relationship between the nodes is determined by the calling relationship between the transaction modules;

[0029] A feature extraction module, used to extract evaluation features of each test case from a plurality of test cases to be sorted; wherein the evaluation features of the test case include: centrality features and network topology features of the transaction module corresponding to the test case in the transaction module network, and attribute features of the test case;

[0030] A prediction module, used for inputting the evaluation features of the plurality of test cases into a pre-trained priority evaluation model to obtain a priority prediction value of each test case generated by the priority evaluation model;

[0031] The sorting module is used to sort the multiple test cases according to the priority prediction values.

[0032] A third aspect of the present application provides an electronic device, including a processor and a memory connected to the processor, wherein:

[0033] The memory is used to store computer programs;

[0034] The processor is used to execute the computer program so that the electronic device implements each step of any one of the above methods.

[0035] A fourth aspect of the present application provides a readable storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device implements the steps of any one of the above methods.

[0036] A fifth aspect of the present application provides a computer program product, comprising computer-readable instructions, which, when executed on an electronic device, enables the electronic device to implement the steps of any one of the above methods.

[0037] It can be seen that the beneficial effects of this application are:

[0038] The test case priority sorting method disclosed in the present application is based on the call relationship of the transaction module in the target software system, constructs the transaction module network of the target software system, extracts the evaluation features of each test case in the multiple test cases to be sorted, inputs the evaluation features of the multiple test cases into the priority evaluation model that has been trained in advance, obtains the priority prediction value of each test case, and then sorts the multiple test cases according to the priority prediction value. Among them, the evaluation features of the test case include the centrality features and network topology features of the transaction module corresponding to the test case in the transaction module network, and the attribute features of the test case. The centrality features of the transaction module corresponding to the test case in the transaction module network can reflect the importance and influence of the role of the test case in the target software system, the network topology features of the transaction module corresponding to the test case in the transaction module network can reflect the complexity of the test case, and the attribute features of the test case are also related to the importance of the test case. The technical solution disclosed in the present application combines the centrality characteristics and network topology characteristics of the transaction module corresponding to the test case in the transaction module network and the attribute characteristics of the test case into evaluation characteristics of the test case. Based on the aforementioned evaluation characteristics, the importance and complexity of each test case can be comprehensively estimated, and then the priority order of multiple test cases can be determined, so that the sorting of test cases is more in line with the test requirements, thereby improving test efficiency and reducing test costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0040] Figure 1 A flowchart of a test case priority sorting method disclosed in this application;

[0041] Figure 2 A schematic diagram of the principle of the grid search hyperparameter method disclosed in this application;

[0042] Figure 3 This is a schematic diagram of the principle of the K-fold cross-validation method disclosed in this application;

[0043] Figure 4 This is a schematic diagram of the principle of the random forest model disclosed in this application;

[0044] Figure 5 A schematic diagram of the principle of the training process of the priority evaluation model disclosed in this application;

[0045] Figure 6 A schematic diagram of the structure of a test case priority sorting device disclosed in this application;

[0046] Figure 7 This is a schematic diagram of the structure of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0047] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0048] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable in appropriate circumstances, which is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the present application. In addition, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only include those elements, but also include other elements that are not clearly listed, or also include elements inherent to such process, method, article or equipment. In the absence of more restrictions, the elements limited by the statement "include one..." do not exclude the existence of other identical elements in the process, method, article or equipment including the elements.

[0049] The present application discloses a test case priority sorting method, device, electronic device and readable storage medium, which determine the execution order of test cases to better meet test requirements, thereby improving test efficiency and reducing test costs.

[0050] See also Figure 1 , Figure 1This is a flow chart of a test case priority sorting method disclosed in this application. The method is executed by an electronic device and includes:

[0051] S101: Build a transaction module network of the target software system.

[0052] In this application, the software system that needs to be regression tested is called the target software system. The target software system includes multiple transaction modules, and each transaction module corresponds to a different test case. The transaction module network of the target software system is a directed unweighted graph, and the nodes in the directed unweighted graph are the transaction modules in the target software system. The relationship between the nodes is determined by the call relationship between the transaction modules.

[0053] Optionally, construct a transaction module network for the target software system using the following solution:

[0054] First, static analysis is performed on the source code of the target software system to obtain the calling relationship between the transaction modules in the target software system;

[0055] Second, a directed unweighted graph is constructed based on the calling relationship between each transaction module in the target software system. The directed unweighted graph is a transaction module network of the target software system.

[0056] In the implementation, the source code of the target software system is obtained, and the source code is scanned and parsed using static code analysis technology to extract the calling relationship between the methods in the target software system. The methods representing transactions usually have specific naming rules and functional characteristics. The extracted methods are identified and screened using keyword matching to obtain the calling relationship between transaction modules. If a transaction module calls the method of another transaction module, the corresponding weights of the two transaction modules are set to 1 in the calling relationship table.

[0057] After obtaining the call relationship between each transaction module in the target software system, a directed unweighted graph is constructed based on the call relationship between each transaction module. Specifically, each transaction module is regarded as a node, and the edge connecting two nodes represents the degree of correlation between the two transaction modules. The weight of the edge represents the size of the association relationship. A weight of 1 indicates that there is a call relationship between the two transaction modules, and a weight of 0 indicates that there is no call relationship between the two transaction modules.

[0058] S102: Extracting evaluation features of each test case from a plurality of test cases to be sorted.

[0059] S103: Inputting the evaluation features of the multiple test cases to be sorted into a pre-trained priority evaluation model to obtain a priority prediction value of each test case output by the priority evaluation model.

[0060] The evaluation characteristics of the test case include: the centrality characteristics and network topology characteristics of the transaction module corresponding to the test case in the transaction module network, and the attribute characteristics of the test case.

[0061] The applicant has found through research that complexity is one of the essential attributes of software, and the topological structure of software will affect its functions, performance, reliability and other indicators. In addition, the complexity of software structure is closely related to the number and distribution of software errors.

[0062] Based on the above findings, in the technical solution disclosed in the present application, the complexity of the test case and the importance of the test case in the target software system are used as the basis for evaluating the priority of the test case. In addition, in the process of evaluating the priority of the test case, the attribute characteristics of the test case itself should also be considered. Therefore, in the technical solution disclosed in the present application, the priority of the test case is evaluated based on the centrality characteristics and network topology characteristics of the transaction module corresponding to the test case in the transaction module network, as well as the attribute characteristics of the test case.

[0063] The centrality characteristics of the transaction module in the transaction module network can reflect the importance and influence of the role of the transaction module in the transaction module network. The transaction module corresponds to the test case, so the centrality characteristics of the transaction module in the transaction module network can reflect the importance and influence of the role of the test case in the target software system. It can be understood that for test cases with higher role importance and greater influence, higher priority should be given, that is, in the regression test of the target software system, the test case should be executed first.

[0064] The centrality features of the transaction module corresponding to the test case in the transaction module network include multiple or all of the following features: degree, clustering coefficient, betweenness, closeness centrality, eigenvector centrality and shortest path length. Optionally, the centrality features include all of the aforementioned features.

[0065] Each feature is described below.

[0066] 1) The degree of a node (i.e., a transaction module) refers to the number of edges connected to the node. In a directed graph, the degree of a node includes in-degree and out-degree. In-degree refers to the number of edges ending at the node, and out-degree refers to the number of edges starting at the node. The degree of a node reflects the connectivity of the node, and highly connected nodes usually have higher degrees.

[0067] 2) The clustering coefficient of a node refers to the ratio of the number of edges actually connected between the node and its neighboring nodes to the number of edges that may be connected. The clustering coefficient of a node reflects the closeness of the node with the network structure around it, that is, the clustering coefficient of a node reflects the role and influence of the node in the network. The larger the clustering coefficient of a node, the more important the role of the node in the network and the higher its influence.

[0068] Optionally, the clustering coefficient of the node is calculated according to formula (1).

[0069] C i =2M i / [k i (k i -1)] Formula (1)

[0070] Among them, C i For node v i The clustering coefficient, M i For node v i The number of edges that actually exist, k i For node v i The number of neighbor nodes, k i (k i -1) is node v i The maximum number of edges that can exist.

[0071] 3) The betweenness of a node refers to the number of times the node appears as an intermediate node in all shortest paths in the network. Nodes with high betweenness play a key role as bridges or information dissemination in the network and have greater influence.

[0072] Optionally, the betweenness of the node is calculated according to formula (2).

[0073]

[0074] Among them, B i For node v i The betweenness, N jl For node v j and node v l The number of shortest paths between jl (i) is node v j and node v l The shortest path between i The number of

[0075] 4) The closeness centrality of a node refers to the reciprocal of the average shortest path length from the node to other nodes in the network. Nodes with high closeness centrality can communicate and interact with other nodes in the network faster.

[0076] Optionally, the closeness centrality of the node is calculated according to formula (3).

[0077]

[0078] Among them, C c (v i ) is the node v i The closeness center of node v is i The number of edges connecting to other nodes, d ij For node v i and node v j The distance between.

[0079] 5) The eigenvector centrality of a node. The importance of a node depends on the number of its neighbor nodes (that is, the degree of the node) and the importance of its neighbor nodes. In other words, the more important the neighbor nodes connected to a node are, the more important the node is.

[0080] Optionally, the closeness centrality of the node is calculated according to formula (4).

[0081]

[0082] Among them, C E (N i ) and x i For node v i The eigenvector centrality, x j For node v j The eigenvector centrality of node v is c, which is the proportional constant. i and node v j When connected, a ij is 1, when node v i and node v j When not connected, a ij is 0.

[0083] 6) The shortest path length refers to the length of the path with the least number of edges connecting two nodes in the network.

[0084] When the regression test occurs in a defect scenario, the shortest path length between other transaction modules and the transaction module where the defect occurs is calculated; when the regression test occurs in multiple defect scenarios, the average of the shortest path lengths between other transaction modules and the transaction module where the defect occurs is taken; when the regression test does not occur in a defect scenario, the feature value is set to 0.

[0085] The network topology features of the transaction module corresponding to the test case in the transaction module network can reflect the complexity of the test case. The network topology features of the transaction module corresponding to the test case in the transaction module network can be extracted in the following way: input the transaction module network into the graph convolutional neural network, and obtain the network topology features of the transaction module from the output of the graph convolutional neural network.

[0086] Graph Convolutional Networks (GCN) applies the neural network model in the field of deep learning to real networks, and can process graphs of networks of arbitrary structures, similar to feature extractors. Graph convolutional neural networks have great advantages in processing graph data and can well capture the complex relationships between nodes. Therefore, in this application, graph convolutional neural networks are used to extract the network topology features of the transaction module in the transaction module network.

[0087] The attribute characteristics of a test case are used to indicate the type of the test case. The types of test cases include, but are not limited to: forward test cases, process test cases, query test cases, test cases with defects in production, life and death test cases, boundary value test cases, abnormal test cases, smoke test cases, black box test cases, white box test cases, core function test cases, user scenario test cases, and compatibility test cases.

[0088] It should be noted that the trained priority evaluation model has the ability to generate priority prediction values ​​close to their actual values ​​based on the evaluation features of the test cases.

[0089] S104: Sort the multiple test cases according to the priority prediction values.

[0090] By executing step S103, priority prediction values ​​of multiple test cases to be sorted are obtained. Multiple test cases are sorted according to the priority prediction values ​​of each test case to determine the execution order of each test case in the regression test of the target software system.

[0091] The test case priority sorting method disclosed in the present application is based on the call relationship of the transaction module in the target software system, constructs the transaction module network of the target software system, extracts the evaluation features of each test case in the multiple test cases to be sorted, inputs the evaluation features of the multiple test cases into the priority evaluation model that has been trained in advance, obtains the priority prediction value of each test case, and then sorts the multiple test cases according to the priority prediction value. Among them, the evaluation features of the test case include the centrality features and network topology features of the transaction module corresponding to the test case in the transaction module network, and the attribute features of the test case. The centrality features of the transaction module corresponding to the test case in the transaction module network can reflect the importance and influence of the role of the test case in the target software system, the network topology features of the transaction module corresponding to the test case in the transaction module network can reflect the complexity of the test case, and the attribute features of the test case are also related to the importance of the test case. The technical solution disclosed in the present application combines the centrality characteristics and network topology characteristics of the transaction module corresponding to the test case in the transaction module network and the attribute characteristics of the test case into evaluation characteristics of the test case. Based on the aforementioned evaluation characteristics, the importance and complexity of each test case can be comprehensively estimated, and then the priority order of multiple test cases can be determined, so that the sorting of test cases is more in line with the test requirements, thereby improving test efficiency and reducing test costs.

[0092] The following is an explanation of the training method of the priority assessment model used in this application.

[0093] A1: Get the training set.

[0094] The training set includes multiple training samples, each training sample includes an evaluation index of a sample test case, and each training sample is annotated with a priority evaluation value. The evaluation features of each sample test case include: the centrality feature and network topology feature of the transaction module corresponding to the sample test case in the transaction module network of the sample software system, and the attribute feature of the sample test case. In addition, each training sample has a label, which is the priority evaluation value of the sample test case.

[0095] The centrality features of the transaction modules corresponding to the sample test cases in the transaction module network of the sample software system include multiple or all of the following parameters: degree, clustering coefficient, betweenness, closeness centrality, eigenvector centrality, and shortest path length. Optionally, the centrality features include all of the aforementioned features.

[0096] It should be noted that the scheme for constructing the transaction module network of the sample software system is similar to the scheme for constructing the transaction module network of the target software system in the previous article, which can be referred to in the previous article. The scheme for extracting the centrality characteristics and network topology characteristics of the transaction modules corresponding to the sample test cases in the transaction module network of the sample software system, as well as the attribute characteristics of the sample test cases, can also be referred to in the previous article.

[0097] A2: Iteratively train the model to be trained based on multiple training samples to optimize the model parameters of the model to be trained.

[0098] During implementation, multiple training samples are input into the model to be trained, and the model to be trained generates a priority prediction value for each training sample. The model to be trained adjusts model parameters based on the priority evaluation values ​​and priority prediction values ​​of the multiple training samples to optimize the model parameters.

[0099] As an optional solution, when a preset end condition is met, the optimal model parameters in multiple rounds of training are obtained, and the model with the optimal model parameters is used as the priority evaluation model. Optionally, the end condition is set to: reaching a preset number of iterations. Optionally, the stop condition is set to: a preset model evaluation indicator reaches a corresponding threshold.

[0100] As another optional solution, after A2, perform the following:

[0101] A3: Get a validation set.

[0102] The validation set includes multiple validation samples, each validation sample includes the evaluation features of a sample test case, and each validation sample is annotated with a priority evaluation value.

[0103] A4: Multiple validation samples are input into the model to be trained. The model to be trained generates a priority prediction value for each validation sample. The hyperparameters are adjusted based on the priority evaluation values ​​and priority prediction values ​​of the multiple validation samples.

[0104] After multiple rounds of training of the model to be trained using the training set, the generalization ability of the model to be trained is evaluated using the validation set, and the overfitting problem is avoided by adjusting the hyperparameters of the model.

[0105] In addition, after the model training is completed, the generalization ability of the model can be evaluated using the test set. The test set should be independent of the training set and the validation set to ensure the fairness and accuracy of the evaluation results.

[0106] The test set includes multiple test samples, each test sample includes the evaluation features of a sample test case, and each test sample is annotated with a priority evaluation value. Multiple test samples are input into the model to be trained, and the model to be trained generates a priority prediction value for each test sample. The value of the model evaluation index is determined based on the priority evaluation values ​​and priority prediction values ​​of the multiple test samples.

[0107] In the above training method, the following scheme can be used to determine the priority evaluation value of the training sample / verification sample / test sample:

[0108] Obtain the execution frequency assessment value, impact scope assessment value, urgency assessment value and defect frequency assessment value of the sample test case; perform weighted summation on the execution frequency assessment value, impact scope assessment value, urgency assessment value and defect frequency assessment value of the sample test case to obtain the priority assessment value of the sample test case.

[0109] The scheme for determining the priority evaluation value of the sample test case can be expressed by formula (5).

[0110]

[0111] Among them, Score is the priority evaluation value of the sample test case, w i is the weight of the ith indicator, x i is the evaluation value of the ith indicator. Table 1 shows the determination method of the evaluation value of each indicator and an example of the weight of each indicator.

[0112] Table 1

[0113]

[0114] Optionally, the model parameters are adjusted by: adopting a grid search parameter adjustment scheme based on cross-validation.

[0115] Grid search parameter tuning is a method to optimize the performance of machine learning models by systematically searching for a given combination of parameters, while cross validation is used to evaluate the performance and generalization ability of machine learning models. This application uses a grid search parameter tuning method based on cross validation to search for the optimal parameters of the model.

[0116] Grid search is to divide the parameter range to be searched into grids, where each grid represents a possible parameter combination. By traversing all grids, the best performing parameters are obtained as the optimal result.

[0117] by Figure 2Take A and B as an example, where A and B are the parameter areas to be searched. When performing grid search, all intersections (a, b) will be taken. In order to obtain the best (a, b), K-fold cross validation is used to evaluate the classification effect of each set of parameters, and the best set of (a, b) is selected as the optimal parameters of the model. The specific steps are as follows:

[0118] 1) Select the parameter range based on experience and set the coordinate network;

[0119] 2) Use K-fold cross validation to evaluate the classification effect. The principle of cross validation is as follows Figure 3 As shown in the figure. First, the original data set is divided into K parts, each subset is used as a test set, and the remaining K-1 parts of data are used as training sets, thus obtaining K models; then, the classification accuracy of the final validation set of the K models is averaged according to the selected parameter values, which is used as the final classification accuracy of the model.

[0120] 3) Traverse the coordinate network, calculate the classification accuracy corresponding to all parameter combinations, and select the parameter combination corresponding to the maximum classification accuracy as the final parameter.

[0121] Optionally, the root mean square error (RMSE), mean absolute error (MAE), and R 2 As a model evaluation indicator. Among them, the root mean square error is used to measure the deviation between the predicted value and the true value; the mean absolute error can better reflect the actual situation of the predicted value error. The smaller the value of the mean absolute error, the closer the predicted value output by the priority evaluation model is to the true value; R 2 It is between 0 and 1. The closer it is to 1, the better the regression fitting effect.

[0122] The calculation method of the root mean square error is shown in formula (6).

[0123]

[0124] Where m is the number of iterations, f i is the predicted value of the sample, y i is the true value of the sample.

[0125] The calculation method of the mean absolute error is shown in formula (7).

[0126]

[0127] Where m is the number of iterations, f i is the predicted value of the sample, y i is the true value of the sample.

[0128] R2 The calculation method of is shown in formula (8).

[0129]

[0130] Where m is the number of iterations, f i is the predicted value of the sample, y i is the true value of the sample, is the mean of the true values ​​of the samples.

[0131] Optionally, based on the above training method, further setting is made: screening the evaluation features used by the model to be trained.

[0132] The feature selection process is the key to building a machine learning model with good training efficiency and generalization performance. The feature selection process is to select the most consistent, relevant and non-redundant feature subset from the initial data set. Good feature selection can not only reduce training time and model complexity, but also ultimately help prevent model overfitting.

[0133] Optionally, the evaluation features used by the priority evaluation model are screened, and the following scheme is adopted: the evaluation features used by the priority evaluation model are screened by adopting an encapsulation method.

[0134] The encapsulation method is a feature selection method that evaluates feature subsets through model prediction indicators, and selects or eliminates several features based on the prediction score each time. The encapsulation method mainly obtains the most important attributes through forward selection and backward elimination. Among them, forward selection uses iterative greedy solution, starting from no features at all, and adding one of the most important features in the remaining feature subset in each iteration. Backward elimination means that all features are used for training initially, and the least important variable is deleted in each iteration, and features are eliminated continuously until the model performance does not improve.

[0135] In addition, the evaluation features used by the priority evaluation model can also be screened by using filtering method or embedding method.

[0136] Figure 5 The principle of the training process of the priority evaluation model in this application is shown.

[0137] As an optional solution, the priority evaluation model adopts a random forest model.

[0138] The random forest model is one of the algorithms for ensemble learning. The principle of the random forest model is as follows Figure 4As shown. The random forest model combines multiple independent decision tree classifiers and makes the final decision by voting or taking the average. Using bootstrap sampling integration or bagging (Bootstrap AGGregatING, referred to as Bagging) for sampling, constructing multiple decision trees, and using unbiased estimation can effectively reduce the sampling error. Therefore, the random forest model has the advantages of high accuracy, strong generalization ability, fast training speed, easy implementation, and avoids overfitting to a certain extent. Therefore, in this application, the random forest model is selected as the priority evaluation model.

[0139] The training process of the priority evaluation model disclosed in the present application is described below with reference to examples. Some transaction modules included in the sample software system are shown in Table 3.

[0140] The dataset is divided into training set, validation set and test set in a ratio of 3:1:1. The training set is used to train the model to optimize the model parameters; the validation set is used to adjust the model hyperparameters; and the test set is used to finally evaluate the model performance. Afterwards, the hyperparameters of the model corresponding to the minimum RMSE and MAE sum are obtained in the validation set through the grid search parameter adjustment method of ten-fold cross-validation, and the test set is trained and tested, and the RMSE, MAE and R are recorded. 2 , and further perform multiple model training through feature recursive elimination (feature screening). Finally, the optimal feature set and model parameters are obtained based on the optimal calculation results of the model.

[0141] The training results are shown in Table 2.

[0142] Table 2

[0143]

[0144]

[0145] In the above training results, the smallest RMSE is 0.1066, and the smallest MAE is 0.2003. The overall values ​​of RMSE and MAE are small. The performance of RMSE and MAE reflects that the error between the predicted value and the actual value of the model is small, and the prediction results of the model are generally close to the actual value. 2 The value is 0.8781, R 2 The overall value of is greater than 0.8, which shows that the model has a strong ability to explain the data, the prediction results fit the actual values ​​well, and the changes in the data are well explained. The model has high accuracy and reliability in the prediction task.

[0146] The sample test cases are sorted according to their predicted priority values ​​to obtain the predicted priority of each test case.

[0147] Table 3

[0148]

[0149]

[0150]

[0151] In Table 3, the actual priority of the sample test cases is well matched with the predicted priority. In actual test scenarios, test cases are usually clearly divided into high, medium, and low levels, but the execution order of test cases at the same level may deviate to a certain extent. Therefore, it is reasonable that the predicted priority and actual priority of some cases in the test set fluctuate in a smaller range.

[0152] The average rate of false detection (APFD) is a quality indicator that measures the effectiveness of a test suite, and can be used to measure the extent to which a test suite can detect software defects. The higher the APFD value, the better the quality of the test suite, and the more effective it is in detecting defects in the software.

[0153] The average false detection rate is calculated as follows:

[0154]

[0155] Where T represents a test case set containing n test cases, m represents the number of defects in the faulty test case set F, and TF i Indicates the position of the test case that first detects the i-th defect in the order. The range of APFD is 0-1. The closer the APFD value is to 1, the faster the error detection speed is and the better the test effect is.

[0156] The present application discloses a test case priority sorting method as mentioned above, and correspondingly, the present application also discloses a test case priority sorting device, and the descriptions of the two in the specification can refer to each other.

[0157] See also Figure 6 , Figure 6 The schematic diagram of the structure of a test case priority sorting device disclosed in the present application includes a network construction module 601 , a feature extraction module 602 , a prediction module 603 and a sorting module 604 .

[0158] The network construction module 601 is used to construct a transaction module network of the target software system. The transaction module network is a directed unweighted graph. The nodes in the directed unweighted graph are transaction modules in the target software system. The relationship between the nodes is determined by the calling relationship between the transaction modules.

[0159] The feature extraction module 602 is used to extract the evaluation features of each test case in the plurality of test cases to be sorted, wherein the evaluation features of the test case include the centrality features and network topology features of the transaction module corresponding to the test case in the transaction module network, and the attribute features of the test case.

[0160] The prediction module 603 is used to input the evaluation features of multiple test cases into a pre-trained priority evaluation model to obtain the priority prediction value of each test case output by the priority evaluation model.

[0161] The sorting module 604 is used to sort the multiple test cases according to the priority prediction values.

[0162] Optionally, the network construction module 601 is specifically used to: perform static analysis on the source code of the target software system to obtain the calling relationship between the transaction modules in the target software system; construct a directed unweighted graph based on the calling relationship between the transaction modules in the target software system, and the directed unweighted graph is the transaction module network of the target software system.

[0163] The test case priority sorting device disclosed in the present application combines the centrality characteristics and network topology characteristics of the transaction module corresponding to the test case in the transaction module network and the attribute characteristics of the test case into evaluation characteristics of the test case. Based on the aforementioned evaluation characteristics, it can comprehensively estimate the importance and complexity of each test case, and then determine the priority order of multiple test cases, so that the sorting of test cases is more in line with the test requirements, thereby improving test efficiency and reducing test costs.

[0164] Optionally, based on the test case priority sorting device disclosed above in the present application, a model training module is further set up.

[0165] The model training module is specifically used for: obtaining a training set, the training set includes multiple training samples, each training sample includes the evaluation characteristics of a sample test case, and each training sample is marked with a priority evaluation value; the evaluation characteristics of each sample test case include: the centrality characteristics and network topology characteristics of the transaction module corresponding to the sample test case in the transaction module network of the sample software system, and the attribute characteristics of the sample test case; iteratively training the model to be trained based on multiple training samples to optimize the model parameters of the model to be trained.

[0166] The model training module is also used to: obtain a validation set, the validation set includes multiple validation samples, each validation sample includes an evaluation feature of a sample test case, and each validation sample is annotated with a priority evaluation value; input multiple validation samples into the model to be trained, the model to be trained generates a priority prediction value for each validation sample, and adjusts hyperparameters based on the priority evaluation values ​​and priority prediction values ​​of the multiple validation samples.

[0167] The model training module is also used to: screen the evaluation features used by the model to be trained.

[0168] Optionally, the priority evaluation model adopts a random forest model.

[0169] It should be noted that the specific technical details of the model training module can be found in the description above.

[0170] The application also discloses an electronic device.

[0171] See also Figure 7 , Figure 7 The hardware structure of the electronic device is shown. The electronic device includes: a processor 701 , a memory 702 , a communication interface 703 , and a communication bus 704 .

[0172] In the embodiment of the present application, the number of the processor 701, the memory 702, the communication interface 703, and the communication bus 704 is at least one, and the processor 701, the memory 702, and the communication interface 703 communicate with each other through the communication bus 704. The communication bus 704 can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0173] It should be noted that those skilled in the art can understand that Figure 7 The structure of the electronic device shown in the figure does not constitute a limitation on the electronic device, and the electronic device may include Figure 7 More or fewer components may be shown, or certain components may be combined, or the components may be arranged differently.

[0174] Combine the following Figure 7 Provide a detailed introduction to the various components of electronic equipment.

[0175] The processor 701 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. It executes various functions of the electronic device and processes data by running or executing software programs and / or modules stored in the memory 702 and calling data stored in the memory 702, thereby monitoring the electronic device as a whole.

[0176] The processor 701 may be a central processing unit (CPU), or an application specific integrated circuit ASIC (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, etc.;

[0177] The memory 702 may include a memory, such as a high-speed random-access memory (RAM) and a read-only memory (ROM), and may also include a large-capacity storage device, such as at least one disk storage.

[0178] The memory 702 stores a computer program, and the processor 701 can call the computer program stored in the memory to enable the electronic device to implement each step of any of the test case priority sorting methods disclosed above.

[0179] The present application also discloses a readable storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device implements the various steps of any one of the test case priority sorting methods disclosed above.

[0180] The present application also discloses a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements each step of any one of the test case priority sorting methods disclosed above.

[0181] It should be noted that the technical features recorded in the various embodiments in the specification can be replaced or combined with each other, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other. The steps in the methods of the various embodiments of the present application can be adjusted in sequence, merged and deleted according to actual needs. The modules in the devices and equipment of the various embodiments of the present application can be merged, divided and deleted according to actual needs.

[0182] In the several embodiments provided in this application, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, for example, multiple modules can be combined or integrated into another module, or some features can be ignored or not executed.

[0183] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. The various technical features in the various embodiments can be arranged and combined to form new embodiments. For the test case prioritization device, electronic device, readable storage medium and computer program product disclosed in the embodiments, since they correspond to the test case prioritization method disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0184] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0185] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

Claims

1. A test case prioritization method, characterized in that: include: Constructing a transaction module network of the target software system, wherein the transaction module network is a directed unweighted graph, the nodes in the directed unweighted graph are transaction modules in the target software system, and the relationship between the nodes is determined by the calling relationship between the transaction modules; Extracting evaluation features of each test case from the plurality of test cases to be sorted; wherein the evaluation features of the test case include: centrality features and network topology features of the transaction module corresponding to the test case in the transaction module network, and attribute features of the test case; Inputting the evaluation features of the plurality of test cases into a pre-trained priority evaluation model to obtain a priority prediction value of each test case generated by the priority evaluation model; The multiple test cases are sorted according to the priority prediction values.

2. The test case prioritization method according to claim 1, characterized in that: The transaction module network of the target software system is constructed, including: Performing static analysis on the source code of the target software system to obtain the calling relationship between the transaction modules in the target software system; A directed unweighted graph is constructed based on the calling relationship between the transaction modules in the target software system, and the directed unweighted graph is a transaction module network of the target software system.

3. The test case prioritization method according to claim 1, characterized in that: The centrality characteristics of the transaction module corresponding to the test case in the target transaction module network include multiple or all of the following characteristics: degree, clustering coefficient, betweenness, closeness centrality, eigenvector centrality and shortest path length.

4. The test case prioritization method according to claim 1, characterized in that: The training process of the priority evaluation model includes: A training set is obtained, wherein the training set includes a plurality of training samples, each training sample includes an evaluation feature of a sample test case, and each training sample is annotated with a priority evaluation value; the evaluation feature of each sample test case includes: a centrality feature and a network topology feature of a transaction module corresponding to the sample test case in a transaction module network of a sample software system, and an attribute feature of the sample test case; The model to be trained is iteratively trained based on the multiple training samples to optimize the model parameters of the model to be trained.

5. The test case prioritization method according to claim 4, characterized in that: The training process of the priority evaluation model further includes: Obtaining a validation set, the validation set comprising a plurality of validation samples, each validation sample comprising an evaluation feature of a sample test case, and each validation sample being annotated with a priority evaluation value; The multiple verification samples are input into the model to be trained, the model to be trained generates a priority prediction value for each verification sample, and hyperparameters are adjusted based on the priority evaluation values ​​and priority prediction values ​​of the multiple verification samples.

6. The test case prioritization method according to claim 5, characterized in that: The training process of the priority evaluation model further includes: Screen the evaluation features used by the model to be trained.

7. The test case prioritization method according to claim 4, characterized in that: The priority evaluation value of any sample test case is determined as follows: Obtaining an execution frequency assessment value, an impact scope assessment value, an urgency assessment value, and a defect frequency assessment value of the sample test case; The execution frequency evaluation value, impact scope evaluation value, urgency evaluation value and defect frequency evaluation value of the sample test case are weightedly summed to obtain the priority evaluation value of the sample test case.

8. A test case priority sorting device, characterized in that: include: A network construction module, used to construct a transaction module network of a target software system, wherein the transaction module network is a directed unweighted graph, the nodes in the directed unweighted graph are transaction modules in the target software system, and the relationship between the nodes is determined by the calling relationship between the transaction modules; A feature extraction module, used to extract evaluation features of each test case from a plurality of test cases to be sorted; wherein the evaluation features of the test case include: centrality features and network topology features of the transaction module corresponding to the test case in the transaction module network, and attribute features of the test case; A prediction module, used for inputting the evaluation features of the plurality of test cases into a pre-trained priority evaluation model to obtain a priority prediction value of each test case generated by the priority evaluation model; The sorting module is used to sort the multiple test cases according to the priority prediction values.

9. An electronic device, characterized in that: comprising a processor and a memory connected to the processor; The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device implements each step of the test case prioritization method as described in any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium carries one or more computer programs, and when the one or more computer programs are executed by an electronic device, the electronic device implements the various steps of the test case prioritization method as described in any one of claims 1 to 7.

Images