Software testing capability evaluation method based on DP-ANN and software defect mode

Abstract

The invention relates to a software testing capability evaluation method based on DP-ANN and a software defect mode, and belongs to the field of software testing. The method comprises the following steps: preprocessing various types of defect data of a software defect model, and constructing a software defect mode; establishing a software testing capability evaluation DP-ANN model of a software defect mode based on a back propagation BP neural network algorithm, and optimizing back propagation gradient calculation; taking the preprocessed data as input, setting an association relationship of each layer and a data volume of each layer through an established DP-ANN model, and training to obtain an optimal weight and a threshold value required by an evaluation model; and applying the final training result to software test capability evaluation of the software defect mode. The method is suitable for calculating the weight of the evaluation system of the software defect mode, the efficiency of large-scale complex software defect evaluation work can be improved, and the objectivity of the weight is improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of software testing, and particularly relates to a software testing capability evaluation method based on DP-ANN and software defect patterns. BACKGROUND

[0002] With the continuous progress of the software industry, the scale of software is also increasingly large and complex. In the face of large-scale and high-complexity software, the amount of software testing work increases synchronously. The larger the scale of software is, the more testing problems there will be. Testing is a measure of software quality. Even for high-complexity software, the quality of software testing is usually evaluated in terms of the types and number of testing problems.

[0003] In order to scientifically and efficiently and fairly evaluate the software testing capability of large-scale and high-complexity software, a suitable model is usually used to quantify the results of software testing. Generally, the software problem category and number, and problem severity are simply divided. In the case of small-scale and low-complexity software, this is a simple and effective way. In the case of large-scale and high-complexity software, a proper and good software defect pattern model needs to be constructed to solve the problem by dividing the software defect pattern.

[0004] Based on large-scale and complex software, after constructing a good software defect pattern, in the evaluation of software testing capability, the subjectivity of the analytic hierarchy process is strong in most evaluation methods. In the case of small-scale software, the analytic hierarchy process is simple, convenient and effective. In the case of such software, the level of the analytic hierarchy process increases, which undoubtedly increases the work in all aspects. In addition, the number of indicators is large, and the weight of each indicator needs to be calculated separately. Moreover, the deeper the level of division is, the less the advantage of the analytic hierarchy process can be reflected. The data envelopment analysis method (DEA) is used to determine each decision making unit (DMU), and each decision making unit has the same input quantity and the same output quantity. In this case, how to determine the input and output needs to be carefully considered. Regardless of how the input and output are determined, the input and output will inevitably be proportional to the scale. The DEA method is very sensitive to the selection and definition of input and output data, and may be difficult to apply to large data sets and complex situations. SUMMARY

[0005] (I) Technical problems to be solved

[0006] The technical problem to be solved by the present application is to provide a software testing capability evaluation method based on DP-ANN and software defect patterns to solve the problem of software testing capability evaluation of large-scale and complex software.

[0007] (II) Technical solutions

[0008] To solve the above technical problems, the present application provides a software testing capability evaluation method based on DP-ANN and software defect patterns, which comprises the following steps:

[0009] S1, pre-process various types of defect data of the software defect model, and construct a software defect pattern;

[0010] S2, establish a software testing capability evaluation DP-ANN model of the software defect pattern based on the BP neural network algorithm of back propagation, and optimize the gradient calculation of back propagation;

[0011] S3, according to the pre-processed data as input, set the correlation of each layer and the data volume of each layer through the established DP-ANN model, and train the optimal weight and threshold required by the evaluation model;

[0012] S4, apply the final training result to the software testing capability evaluation of the software defect pattern, use the optimal weight and threshold trained to form an evaluation model, input the result obtained by actual testing into the trained evaluation model, obtain an evaluation value, compare it with the actual observation value, establish the evaluation result level according to the final output value, and perform result analysis and discussion.

[0013] (Three) beneficial effects

[0014] The present application provides a software testing capability evaluation method based on DP-ANN and software defect patterns, which is suitable for the calculation of the weight of the evaluation system of the software defect pattern, can improve the efficiency of large-scale complex software defect evaluation work, and increase the objectivity of the weight. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 The flowchart of the software testing capability evaluation method based on DP-ANN and software defect patterns of the present application;

[0016] Figure 2 The schematic diagram of the software defect model;

[0017] Figure 3 The schematic diagram of the DP-ANN model;

[0018] Figure 4 The running result diagram of the initialization and reset type software defect;

[0019] Figure 5 The mean square error diagram;

[0020] Figure 6 The training state diagram. DETAILED DESCRIPTION

[0021] In order to make the purpose, content and advantages of the present application more clear, the specific embodiments of the present application are described in further detail below in combination with the drawings and examples.

[0022] The present application aims to solve the above-mentioned problems in the background art to improve the fairness, impartiality and effectiveness of the evaluation system, and the specific implementation scheme is as follows:

[0023] The present application provides a software testing capability evaluation method based on DP-ANN and software defect mode, which comprises the following steps:

[0024] S1, pre-processing various types of defect data of software defect model, constructing software defect mode;

[0025] In step S1, the data is divided according to the defect type, pre-processed by principal component analysis method, cluster analysis method, etc., and the software defect mode is constructed, which specifically comprises the following steps:

[0026] S11, record the software defect mode (Software Defect Mode) as SDM, the software defect sub-mode (Software Defect Sub-Mode) as SDSM, and the software defect (Software Defect) as SD, the i-th software defect is recorded as SD i , the j-th software defect sub-mode is recorded as SDSM j .

[0027] S12, classify the software defects uniformly, if there are J sub-modes, the j-th software defect sub-mode has k types of software defects, then the relationship is SD i ∈SDSM j , i=1,2,…,k, wherein, SD i does not belong to the sub-mode other than SDSM j , for example, as shown in Table 1.

[0028] Table 1 Software defect example

[0029]

[0030]

[0031] For the classification and management of software defect modes such as initialization and reset class, calculation and algorithm class, logic design class, data processing class, interrupt timing design class, bus communication class, security design class, memory related class, programming language specification class, according to the software characteristics, it is divided into optional, optional, conditional selection, and the description is made. As shown in Table 2.

[0032] Table 2 Defect mode classification

[0033]

[0034] Defect sub-pattern is used to describe a relatively specific defect description, which is a state between software defect pattern and software defect. It usually contains a plurality of specific software defects. The relationship between software defect sub-pattern and software defect pattern is SDSM j ∈SDM, j = 1, 2, … J. The software defect sub-pattern classification is shown in Table 3.

[0035] Table 3 Defect sub-pattern classification

[0036]

[0037]

[0038] S2, establishing a software test capability evaluation DP-ANN model of software defect pattern based on BP neural network algorithm of back propagation, and optimizing gradient calculation of back propagation;

[0039] In the step S2, the perceptron in the ANN model is a mathematical model that simulates the neurons of living beings in real life. In biology, neurons, i.e. neuron cells, are the most basic structural and functional units of the nervous system, and have the function of contacting and integrating input information and transmitting information. The neuron processes include a large number of dendrites and a small number of axons. The dendrites have the function of receiving impulses transmitted by the axons of other neurons and transmitting them to the cell body. The axons have the function of receiving external stimuli, and the synapses of the axons of the neurons are connected to other neurons. The neurons generally have an activated state and a non-activated state. When in the activated state, the neurons can emit electrical pulses and transmit them to other neurons along the two types of synapses.

[0040] This step includes:

[0041] S21, determining the activation function of the DP-ANN model, which is set to the softsign function here, and can also be other activation functions:

[0042]

[0043] S22, constructing the input layer, the hidden layer and the output layer of the corresponding BP neural network according to the software defect model, wherein the input layer is the software defect pattern, the hidden layer is the software defect sub-pattern, and the output layer is the software test capability evaluation. For convenience of representation, the input layer is denoted as The first layer has n perceptrons, and the output layer is denoted as output. Assuming that the model has a plurality of layers, let is the value of the input transmitted by the i-th sensor of the l-1-th layer, The weight of the jth sensor in the lth layer to the input from the ith sensor in the (l-1)th layer, The output value of the jth sensor in the lth layer after transformation by the activation function, l The number of sensors in the lth layer, The bias of the jth sensor in the lth layer to all inputs from the previous layer; the summation output of the jth sensor in the lth layer is

[0044]

[0045] Then

[0046] S23, the actual jth result obtained by training Compared with the expected result Set the formula of the mean square error function as:

[0047]

[0048] S24, in order to obtain an objective and reasonable weight, the error obtained in S23 needs to be adjusted. According to the commonly used gradient descent method, the loss function can be continuously reduced, and the weight is updated in reverse. In order to speed up the gradient descent, the dynamic programming idea is introduced. Taking the calculation from the hidden layer to the output layer as an example (the principle from the input layer to the hidden layer is consistent), the learning rate η is determined, and a new η can be replaced every time the calculation is performed, and finally the DP-ANN model is established. The specific process is as follows:

[0049] S241, according to the basis of S23, the update of the parameters is as follows

[0050]

[0051] S242, then according to formula (4), the partial derivative of about can be obtained:

[0052]

[0053] According to the partial derivative of to , the following can be obtained:

[0054]

[0055] S243, let be Then

[0056] S244. The software defect model established in S1 is suitable for the backpropagation neural network of ANN. Furthermore, based on the software defect model, the correlation between each layer can be analyzed. This is very suitable for breaking down a problem into several smaller problems, solving them, and then connecting them through their correlations to form the answer to the original problem. For the software defect model, based on the idea of ​​dynamic programming (DP), a problem is decomposed into several subproblems. The optimal solution is found for each subproblem, and the optimal solution of the previous subproblem serves as the basis for solving the next subproblem step by step. Finally, the last subproblem is the optimal solution to the original problem.

[0057]

[0058] because It is the output value of the j-th perceptron in its hierarchy after nonlinear transformation by the activation function softsign(x). Differentiation yields:

[0059]

[0060] S245. The final DP-ANN model is as follows:

[0061]

[0062] S3. Using the preprocessed data as input, the DP-ANN model is established, and the relationships and data volume of each layer are set to train and obtain the optimal weights and thresholds required to evaluate the model.

[0063] S4. Apply the final training results to the software defect pattern software testing capability assessment. Use the optimal weights and thresholds obtained during training to form an assessment model. Input the results obtained from actual testing into the trained assessment model to obtain the assessment value. Compare this value with the actual observed values. Based on the final output value, establish the assessment result level, and conduct result analysis and discussion. The mean absolute error (MAE), mean squared error (MSE), and root mean square error (RMSE) obtained during training are used as indicators to judge the difference between the DP-ANN model's assessment value and the actual observed values, and are used to determine the degree of fit of the assessment model on the given data.

[0064] Here, the final output is categorized as Excellent or Good. The characteristics are the expected proportion and number of software defects at different detection stages and severity levels under the constructed software defect pattern, within the corresponding software scale. This data is used as training data to train the evaluation model. When different testing organizations participate in a project, the test results from each organization are used as predictive input, combined with the trained evaluation model, to obtain the Excellent and Good evaluation capability results.

[0065] Embodiment 1:

[0066] The specific implementation is described in detail below in combination with the drawings. In order to simplify the data scale, the initialization and reset of the software defect sub-mode are taken as examples for solving the sub-problems according to the idea of dynamic programming.

[0067] A software testing capability evaluation method based on DP-ANN and software defect mode, the method comprising the following steps, see Figure 1 :

[0068] a. Preprocessing various types of defect data of the software defect model, constructing a software defect mode, see Figure 2 ;

[0069] b. Establishing a software testing capability evaluation model of the software defect mode based on the neural network algorithm of back propagation, optimizing the gradient calculation of back propagation;

[0070] For step a, the data is divided according to the defect type, preprocessed by principal component analysis method, cluster analysis method, etc., and the software defect mode is constructed, characterized in that it comprises the following steps:

[0071] A1 Software defect mode (Software Defect Mode) is SDM, software defect sub-mode is SDSM (Software Defect Sub-Mode), software defect (Software Defect) is SD, the i-th software defect is denoted as SD i , the j-th software defect sub-mode is denoted as SDSM j .

[0072] A2 The software defects are classified uniformly, if there are n sub-modes in total, the j-th software defect sub-mode has k types of software defects, then the relationship is SD i ∈SDSM j , i = 1, 2, …, k, wherein SD i does not belong to the sub-mode other than SDSM j .

[0073] Table 1 Software defect example

[0074]

[0075]

[0076] The software defect modes are classified and managed according to initialization and reset class, calculation and algorithm class, logic design class, data processing class, interrupt timing design class, bus communication class, security design class, memory related class, programming language specification class and the like, and are classified into optional, selectable and conditional selection according to software characteristics, and are described. As shown in Table 2.

[0077] Table 2 defect mode classification

[0078]

[0079]

[0080] The defect sub-mode is used to describe a relatively specific defect description, which is a state between the software defect mode and the software defect. It usually contains a plurality of specific software defects. The relationship between the software defect sub-mode and the software defect mode is SDSM j ∈SDM, j = 1, 2, … The software defect sub-mode classification is shown in Table 3.

[0081] Table 3 defect sub-mode classification

[0082]

[0083]

[0084] According to the software defect model established in a, for step b, the sensor is a mathematical model that simulates the neurons of living beings in real life. In biology, neurons, also known as neuron cells, are the most basic structural and functional units of the nervous system, and have the function of contacting and integrating input information and transmitting information. The neuron processes include dendrites and axons, the dendrites are numerous and have the function of receiving impulses from other neuron axons and transmitting them to the cell body, the axons are few and have the function of receiving external stimuli, and the synapses of the neuron axons are connected with other neurons. The neuron generally has an activated state and a non-activated state, and when in the activated state, it can emit an electric pulse and transmit it to other neurons along the two types of synapses.

[0085] The method comprises:

[0086] B1 as shown in Figure 3 , a DP-ANN neural network model is constructed, and the activation function of the DP-ANN model is determined, and the activation function is set to a softsign function here, and other activation functions can also be used:

[0087]

[0088] B2 According to the software defect model, the input layer, hidden layer and output layer of the corresponding BP neural network are constructed, wherein the input layer is the software defect mode, the hidden layer is the software defect sub-mode, and the output layer is the software test capability evaluation. For convenience of representation, the input layer is denoted as is the first layer, and there are n perceptrons. The output layer is denoted as output, which is the target layer. It is assumed that the model has a plurality of layers. Let be the value of the input from the i-th sensor of the l-1-th layer, be the weight of the j-th sensor of the l-th layer to the i-th input of the l-1-th layer, be the output value of the j-th sensor of the l-th layer after being transformed by the activation function, and a l be the number of sensors of the l-th layer, be the bias of the j-th sensor of the l-th layer to all inputs of the previous layer; and the summation output of the j-th sensor of the l-th layer is

[0089]

[0090]

[0091] B3 The actual j-th result obtained through training is compared with the expected result , and then the formula of the mean square error function is set as:

[0092]

[0093] B4 In order to obtain an objective and reasonable weight, the error obtained in B3 needs to be adjusted. According to the commonly used gradient descent method, the loss function can be continuously reduced, and the weight is updated in reverse. In order to accelerate the gradient descent, the dynamic programming idea is introduced. Taking the calculation from the hidden layer to the output layer as an example (the principle from the input layer to the hidden layer is consistent), the characteristic is that the learning rate η is determined, and a new η can be selected for each calculation. The specific process is as follows:

[0094] C1 According to the basis of 3, the update of the parameter is as follows

[0095]

[0096] C2 According to (formula 3), the partial derivative of with respect to can be obtained:

[0097]

[0098] According to in B2, the partial derivative of is taken with respect to , and the following can be obtained:

[0099]

[0100] C3 For Then

[0101] C4Due to the software defect model established by a, the BP neural network suitable for ANN, in addition to the software defect model, the correlation between each layer can be analyzed, which is very suitable for cutting the problem into several small problems, solving them, and then stringing them up according to their correlation to form the answer to the original problem. According to the idea of dynamic programming (DP), a problem is decomposed into several sub-problems, and the optimal solution of each sub-problem is obtained. The optimal solution of the previous sub-problem is provided to the next sub-problem as a basis, and each sub-problem is solved step by step. Finally, the last sub-problem is the optimal solution of the original problem.

[0102]

[0103] Because is the output value of the jth perceiver in its own layer after non-linear transformation by the activation function softsign(x), and the derivative can be obtained as:

[0104]

[0105] C5The final DP-ANN mathematical model is:

[0106]

[0107] c. Here, the initialization and reset of the software defect sub-mode are taken as an example to solve the sub-problem. According to the software defect model constructed under the corresponding scale, the number of software defects at different detection stages and different hazard levels is taken as a feature input. The software defect hazard level is marked as 1, 2, and 3 according to low, medium, and high. The detection stage static analysis, code review, and dynamic testing are marked as 1, 2, and 3. The declaration and definition of the software defect sub-mode are marked as 0 and 1 respectively, and the division of testing ability is marked as excellent and good, which are represented by 1 and 0 respectively. The experiment is carried out. Different software defects in the same software defect sub-mode need to be summarized. Through training data, the model is obtained, the testing data obtained by different evaluation agencies are simulated, and the software testing ability result is obtained.

[0108] According to the preprocessed data as training input data, as shown in Table 4,

[0109] Table 4 Training input data example

[0110]

[0111] After reading the training input data and output data, setting the prediction input data, i.e. the evaluation results of the three evaluation units, the following data matrix is obtained:

[0112]

[0113] Setting the prediction output data obtains the following data matrix:

[0114]

[0115] The number of nodes of the hidden layer is 2, because the initialization and reset classes are divided into initialization classes and declaration definition classes, the optimal weight and threshold required by the evaluation model are obtained by establishing the evaluation model based on DP-ANN, preprocessing data and training, the result is as shown in Figure 4 , the mean square error is as shown in Figure 5 , and the training state is as shown in Figure 6 .

[0116] d. The final calculation result is applied to the software testing capability evaluation of the software defect mode. The actual obtained result is 0, 1, 0; that is, the first and third are good, and the second is excellent, as shown in Figure 4 .

[0117] The average absolute error MAE of the operation is 0.16663, the mean square error MSE is 0.08326, and the root mean square error RMSE is 0.28855, so it can be concluded that the training result is good. By solving the sub-problem, the data is further expanded to the entire model, and the software testing capability is evaluated.

[0118] The present application provides a software testing capability evaluation method based on DP-ANN and software defect mode, which is suitable for the calculation of the evaluation system weight of the software defect mode, improves the efficiency of large-scale complex software defect evaluation work, and increases the objectivity of the weight.

[0119] The above only describes the preferred embodiments of the present application, and it should be noted that for ordinary skilled persons in the technical field, some improvements and modifications can be made without departing from the technical principles of the present application, and these improvements and modifications should also be considered as the protection scope of the present application.

Claims

1. A software testing capability evaluation method based on DP-ANN and software defect patterns, characterized in that, The method includes the following steps: S1. Preprocess various types of defect data in the software defect model to construct a software defect pattern; S2. Establish a DP-ANN model for evaluating software testing capabilities based on the backpropagation BP neural network algorithm for software defect patterns, and optimize the gradient calculation of backpropagation. S3. Based on the preprocessed data, use it as input, and through the established DP-ANN model, set the correlation of each layer and the amount of data in each layer, and train to obtain the optimal weights and thresholds required to evaluate the model. S4. Apply the final training results to the software testing capability assessment of the software defect pattern. Use the optimal weights and thresholds obtained from the training to form an assessment model. Input the results obtained from the actual test into the trained assessment model to obtain the assessment value. Compare it with the actual observation value. Based on the final output value, establish the assessment result level and conduct result analysis and discussion.

2. The software testing capability evaluation method based on DP-ANN and software defect patterns as described in claim 1, characterized in that, In step S1, the data is divided according to the type of defect, and preprocessed using principal component analysis and cluster analysis to construct a software defect model.

3. The software testing capability evaluation method based on DP-ANN and software defect patterns as described in claim 2, characterized in that, S1 includes: S11. Let the software defect pattern be SDM, the software defect sub-pattern be SDSM, and the software defect be SD. The i-th type of software defect is denoted as SD. i The j-th software defect sub-pattern is denoted as SDSM. j ; S12. Classify software defects uniformly. If there are J seed patterns in total, and the j-th type of software defect sub-pattern has k types of software defects, then the relationship is SD i ∈SDSM j , i = 1, 2, ..., k, where SD i Not part of SDSM j Other sub-patterns; the relationship between the software defect sub-pattern and the software defect pattern is as follows: SDSM j ∈SDM, j=1,2,…J.

4. The software testing capability evaluation method based on DP-ANN and software defect patterns as described in claim 3, characterized in that, Software defect patterns include: initialization and reset, calculation and algorithm, logic design, data processing, interrupt timing design, bus communication, security design, memory-related, and programming language specification. Different software defect patterns are classified and managed, and according to the characteristics of the software, they are divided into mandatory, optional, and conditionally selectable, with explanations provided.

5. The software testing capability evaluation method based on DP-ANN and software defect patterns as described in claim 3 or 4, characterized in that, S2 includes: S21. Determine the activation function for the DP-ANN model; S22. Construct the input layer, hidden layer, and output layer of the corresponding BP neural network based on the software defect model. The input layer represents the software defect pattern, the hidden layer represents the software defect sub-pattern, and the output layer represents the software testing capability assessment. The input layer is denoted as... The first layer consists of n perceptrons, and the output layer is denoted as output, which is the target layer. Assume the model has a total of several layers. The input value is transmitted from the i-th sensor in the (l-1)-th layer. Let be the weight of the input from the j-th sensor in layer l to the ith sensor in layer (l-1). Let a be the output value of the j-th sensor in layer l after transformation by the activation function. l The number of sensors in the l-th layer. The bias from the j-th sensor in layer l with respect to all inputs from the previous layer is: The summation output of the j-th sensor in layer l is: but S23, The actual j-th result obtained through training. Expected results For comparison, the formula for setting the mean square error function is: S24. Adjustments are made based on the error obtained in S23. The loss function is continuously reduced using the gradient descent method, and the weights are updated in reverse. At the same time, in order to accelerate gradient descent, the dynamic programming (DP) idea is introduced, and finally the DP-ANN model is established.

6. The software testing capability evaluation method based on DP-ANN and software defect patterns as described in claim 5, characterized in that, In step S21, the activation function is set to the softsign function:

7. The software testing capability evaluation method based on DP-ANN and software defect patterns as described in claim 6, characterized in that, S24 specifically includes: S241. Based on S23, the parameter updates are as follows: S242, then according to formula (4), we can obtain the following about Partial derivatives: According to S22 right Taking the partial derivative, we get: S243, Order for but S244. Regarding the software defect model, based on the idea of ​​dynamic programming (DP), a problem is decomposed into several subproblems. The optimal solution is found for each subproblem, and the optimal solution of the previous subproblem is used as a basis for solving the next subproblem step by step. Finally, the last subproblem is the optimal solution to the original problem. because It is the output value of the j-th perceptron in its hierarchy after nonlinear transformation by the activation function softsign(x). Differentiation yields: S245. The final DP-ANN model is as follows:

8. The software testing capability evaluation method based on DP-ANN and software defect patterns as described in claim 7, characterized in that, The final output is categorized as excellent or good.

9. The software testing capability evaluation method based on DP-ANN and software defect patterns as described in claim 7, characterized in that, In step S4, the proportion and number of software defects that should exist under different detection stages and different hazard levels under the corresponding software scale are used as features. These data are used as training data to train an evaluation model. When different testing institutions participate in a project, the test results of each testing institution are used as predictive inputs. Combined with the trained evaluation model, excellent and good evaluation capability results are obtained.

10. The software testing capability evaluation method based on DP-ANN and software defect patterns as described in claim 7, characterized in that, The S4 further includes: the mean absolute error (MAE), mean square error (MSE), and root mean square error (RMSE) obtained from training as indicators to judge the difference between the evaluation value of the DP-ANN model and the actual observed value, and to judge the degree of fit of the evaluation model on the given data.

Images