Software testing capability evaluation method based on DP-ANN and software defect mode
The DP-ANN and software defect pattern modeling method enhances the evaluation of software testing capability in large and complex systems by optimizing gradient calculations and determining optimal weights, addressing inefficiencies in existing subjective methods.
Patent Information
- Application Number
- CN202510407906.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2024-08-09
- Filing Date
- 2025-04-02
- Publication Date
- 2025-07-15
AI Technical Summary
The existing software testing capability evaluation methods have problems such as strong subjectivity, low efficiency and difficulty in objective evaluation in large-scale complex software. Especially when using hierarchical analysis method and data envelope analysis method, they cannot be effectively applied to large and complex software.
The evaluation method based on DP-ANN and software defect mode is adopted, and the software defect mode is constructed by preprocessing defect data, and the DP-ANN model based on backpropagation BP neural network algorithm is established, gradient calculation is optimized, optimal weights and thresholds are trained, and the evaluation model is formed, and software testing ability evaluation is carried out.
It improves the efficiency and objectivity of large-scale complex software defect assessments, and enhances the fairness and accuracy of evaluation results.
Smart Images
Figure CN120315985A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of software testing, and particularly relates to a software testing ability evaluation method based on DP-ANN and software defect patterns. Background Art
[0002] With the continuous progress of the software industry, the scale of software has become larger and larger, and the complexity has become higher and higher. In the face of large-scale and highly complex software, the amount of software testing work has increased synchronously. The larger the scale of the software, the more proportional the testing problems will be. Testing is a measure of software quality. Even in the face of highly complex software, the evaluation of the work quality of software testing usually examines aspects such as the types of testing problems and the number of testing problems.
[0003] In order to scientifically, efficiently and fairly evaluate the software testing ability of large-scale and highly complex software, it is usually necessary to quantify the results of software testing with a suitable model. Generally, it will be simply divided by the software problem category and number, and the problem severity. In the face of software with small scale and low complexity, this is a simple and effective way. In large-scale complex software, it is necessary to construct a proper and good model hierarchy of software defect patterns to solve the problem by dividing software defect patterns.
[0004] Based on large-scale complex software, after constructing a good software defect pattern, when evaluating the software testing ability, in most evaluation methods, the analytic hierarchy process (AHP) has strong subjectivity. When the software scale is small, it is simple, convenient and effective. When encountering such software, the levels in the AHP will increase continuously, which will undoubtedly increase the work in all aspects. In addition, the number of indicators is numerous, and the weight of each indicator needs to be calculated separately. Moreover, the deeper the divided levels, the less the advantages of the AHP can be reflected; using the data envelopment analysis method (DEA, Data Envelopment Analysis), each decision-making unit (DMU) is determined, and each decision-making unit has the same input quantity and the same output quantity. In this case, how to determine the input and output needs to be carefully considered. No matter how it is determined, the final input and output will surely be proportional to the scale. The DEA method is very sensitive to the selection and definition of input and output data, and it may be difficult to apply for large data sets and complex situations. Summary of the Invention
[0005] (1) Technical Problems to be Solved
[0006] The technical problem to be solved by the present invention is how to provide a software testing ability evaluation method based on DP-ANN and software defect patterns to solve the problem of evaluating the software testing ability of large-scale complex software.
[0007] (2) Technical Solutions
[0008] To solve the above technical problems, the present invention proposes a software test ability evaluation method based on DP-ANN and software defect patterns. The method includes the following steps:
[0009] S1. Preprocess various types of defect data of the software defect model to construct software defect patterns;
[0010] S2. Establish a software test ability evaluation DP-ANN model for software defect patterns based on the BP neural network algorithm of backpropagation, and optimize the gradient calculation of backpropagation;
[0011] S3. According to the preprocessed data, as input, through the established DP-ANN model, set the association relationship of each layer and the data volume of each layer, and train to obtain the optimal weights and thresholds required for the evaluation model;
[0012] S4. Apply the final training result to the software test ability evaluation of software defect patterns. Use the optimal weights and thresholds obtained by training to form an evaluation model. Input the results obtained from actual tests into the trained evaluation model to obtain an evaluation value. Compare it with the actual observed value. According to the final output value, establish the evaluation result level and conduct result analysis and discussion.
[0013] (3) Beneficial effects
[0014] The present invention proposes a software test ability evaluation method based on DP-ANN and software defect patterns. The present invention proposes a software test ability evaluation method based on DP-ANN and software defect patterns, which is applicable to the calculation of the weights of the evaluation system of software defect patterns, can improve the efficiency of large-scale complex software defect evaluation work, and increase the objectivity of the weights. Description of the drawings
[0015] Figure 1 It is a flow chart of the software test ability evaluation method based on DP-ANN and software defect patterns of the present invention;
[0016] Figure 2 It is a schematic diagram of the software defect model;
[0017] Figure 3 It is a schematic diagram of the DP-ANN model;
[0018] Figure 4 It is a running result diagram of initialization and reset type software defects;
[0019] Figure 5 It is a mean square error diagram;
[0020] Figure 6 It is a training status diagram. Detailed implementation manners
[0021] To make the objectives, content, and advantages of the present invention clearer, the following further describes in detail the specific implementation manners of the present invention in conjunction with the accompanying drawings and embodiments.
[0022] The present invention aims to solve the problems existing in the above background to improve the fairness, impartiality, and effectiveness of the evaluation system. The specific implementation plan is as follows:
[0023] The present invention provides a software test ability evaluation method based on DP-ANN and software defect patterns. The method includes the following steps:
[0024] S1. Preprocess various types of defect data of the software defect model, and construct software defect patterns;
[0025] In the step S1, the data is divided according to the defect types, and preprocessed through principal component analysis, clustering analysis, etc. to construct software defect patterns. The specific steps include the following:
[0026] S11. Denote the software defect mode (Software DefectMode) as SDM, the software defect sub-mode as SDSM (Software Defect Sub-Mode), the software defect (Software Defect) as SD, and the i-th type of software defect as SD i , and the j-th type of software defect sub-mode is denoted as SDSM j .
[0027] S12. Classify the software defects uniformly. If there are J sub-modes in total, and the j-th type of software defect sub-mode has k types of software defects, then the relationship is SD i ∈SDSM j , i = 1, 2,..., k. Among them, SD i does not belong to the sub-modes other than SDSM j . An example is shown in Table 1.
[0028] Table 1 Software defect example
[0029]
[0030]
[0031] Classify and manage software defect patterns such as initialization and reset, calculation and algorithm, logic design, data processing, interrupt timing design, bus communication, security design, memory-related, programming language specification, etc. According to the software characteristics, they are divided into mandatory, optional, and conditional selection, and are explained. As shown in Table 2.
[0032] Table 2 Defect mode classification
[0033]
[0034] The defect sub - pattern is used to describe a relatively specific defect description, which is a state between the software defect pattern and the software defect. It usually contains multiple specific software defects. The relationship between the software defect sub - pattern and the software defect pattern is that SDSM j ∈SDM, j = 1, 2, … J. The classification of software defect sub - patterns is shown in Table 3 below.
[0035] Table 3 Classification of Defect Sub - patterns
[0036]
[0037]
[0038] S2. Establish a software test ability evaluation DP - ANN model for software defect patterns based on the back - propagation BP neural network algorithm, and optimize the gradient calculation of back - propagation;
[0039] In the step S2, in the ANN model, the perceptron is a mathematical model that mimics the neurons of living organisms in real life. In biology, a neuron is a neuron cell, which is the most basic structure and functional unit of the nervous system, and has the function of connecting and integrating input information and transmitting information. Neuron processes include dendrites and axons. There are many dendrites, whose function is to receive impulses from the axons of other neurons and transmit them to the cell body. There are few axons, whose function is to receive external stimuli. The synapses of the axons of neurons will connect to other neurons. Neurons generally have an active state and an inactive state. When in the active state, they can emit electrical pulses and transmit them to other neurons along two types of synapses.
[0040] This step includes:
[0041] S21. Determine the activation function of the DP - ANN model. Here, the activation function is set to the softsign function, and it can also be other activation functions:
[0042]
[0043] S22. Construct the input layer, hidden layer, and output layer of the corresponding BP neural network according to the software defect model. Among them, the input layer is the software defect pattern, the hidden layer is the software defect sub - pattern, and the output layer is the software test ability evaluation. For convenience of representation, the input layer is denoted as as the first layer, with a total of n perceptrons. The output layer is denoted as output as the target layer. Assume that the model has several layers, and let be the value input from the i - th sensor of the (l - 1) - th layer, $w_{ij}^l$ is the weight of the $j$-th sensor in the $l$-th layer for the input from the $i$-th sensor in the $(l - 1)$-th layer. $a_j^l$ is the output value after transformation by the activation function from the $j$-th sensor in the $l$-th layer. l $n_l$ is the number of sensors in the $l$-th layer. $b_j^l$ is the bias for all inputs from the previous layer to the $j$-th sensor in the $l$-th layer; the sum output of the $j$-th sensor in the $l$-th layer is
[0044]
[0045] Then
[0046] S23. The actual $j$-th result obtained through training is compared with the expected result and the formula for setting the mean square error function is:
[0047]
[0048] S24. To obtain an objective and reasonable weight, adjustments need to be made based on the error obtained in S23. The common gradient descent method can be used to continuously reduce the loss function and update the weight in the reverse direction. To accelerate the gradient descent, the idea of dynamic programming is introduced. Taking the calculation from the hidden layer to the output layer as an example (the principle from the input layer to the hidden layer is the same), the learning rate $\eta$ is determined, and $\eta$ can be changed anew for each calculation. Finally, a DP-ANN model is established. Specifically as follows:
[0049] S241. Based on S23, the update of the parameters is as follows
[0050]
[0051] S242. Then, according to formula (4), the partial derivative of can be obtained:
[0052]
[0053] According to in S22, taking the partial derivative of with respect to
[0054]
[0055] S243. Let be Then
[0056] S244. Since the software defect model established in S1 is applicable to the BP neural network of ANN, in addition, according to the software defect model, the correlation between each layer can be analyzed, which is very suitable for cutting the problem into several small problems. After solving them, they are strung together through their correlation to form the answer to the original problem. For the software defect model, according to the idea of dynamic programming (DP), a problem is decomposed into several sub-problems, and the optimal solution is found for each sub-problem. The optimal solution of the previous sub-problem is provided to the next sub-problem as a basis, and each sub-problem is solved step by step. Finally, the last sub-problem is the optimal solution to the original problem.
[0057]
[0058] because is the output value of the jth perceptron in its layer after nonlinear transformation by the activation function softsign(x). Taking the derivative we get:
[0059]
[0060] S245. The DP-ANN model finally established is:
[0061]
[0062] S3. Based on the preprocessed data, as input, through the established DP-ANN model, set the association relationship of each layer, the amount of data of each layer, and train to obtain the optimal weights and thresholds required for the evaluation model.
[0063] S4. Apply the final training results to the software testing capability assessment of software defect patterns. Use the optimal weights and thresholds obtained through training to form an assessment model. The results obtained through actual testing are input into the trained assessment model to obtain an assessment value, which is compared with the actual observed value. According to the final output value, the assessment result level is established, and the results are analyzed and discussed. The mean absolute error MAE, mean square error MSE, root mean square error RMSE, etc. obtained through training are used as indicators to judge the difference between the evaluation value of the DP-ANN model and the actual observed value, and are used to judge the degree of fit of the assessment model on the given data.
[0064] Here, the final output is divided into excellent and good. Based on the constructed software defect patterns, the proportion and quantity of software defects that should exist under corresponding software scales, different detection stages, and different hazard levels are used as features. These data are used as training data to train an evaluation model. When different evaluation agencies participate in a project, for the test results of each evaluation agency, as the prediction input, combined with the trained evaluation model, the evaluation ability results of excellent and good are obtained.
[0065] Embodiment 1:
[0066] The following is a detailed description of the specific implementation manner in conjunction with the accompanying drawings. To simplify the data scale, taking the initialization and reset software defect sub-pattern as an example of solving sub-problems according to the idea of dynamic programming.
[0067] A software test ability evaluation method based on DP-ANN and software defect patterns, the method includes the following steps, see Figure 1 :
[0068] a. Preprocess various types of defect data of the software defect model, and construct a software defect pattern, see Figure 2 ;
[0069] b. Establish a software test ability evaluation model for the software defect pattern based on the neural network algorithm of backpropagation, and optimize the gradient calculation of backpropagation;
[0070] For step a, the data is divided according to the defect types, and preprocessed through methods such as principal component analysis and clustering analysis to construct a software defect pattern. It is characterized in that it specifically includes the following steps:
[0071] A1 Denote the software defect mode as SDM, the software defect sub-mode as SDSM (Software Defect Sub-Mode), the software defect as SD, and the i-th type of software defect as SD i , and the j-th type of software defect sub-mode is denoted as SDSM j .
[0072] A2 Classify the software defects uniformly. If there are n sub-modes in total, and the j-th type of software defect sub-mode has k types of software defects, then the relationship is SD i ∈SDSM j , i = 1, 2,..., k, where SD i does not belong to the sub-modes other than SDSM j .
[0073] Table 1 Software Defect Example
[0074]
[0075]
[0076] Classify and manage software defect patterns such as initialization and reset classes, calculation and algorithm classes, logic design classes, data processing classes, interrupt timing design classes, bus communication classes, security design classes, memory-related classes, programming language specification classes, etc. According to the software characteristics, they are divided into mandatory, optional, and conditional selection, and explanations are provided. As shown in Table 2.
[0077] Table 2 Defect Pattern Classification
[0078]
[0079]
[0080] A defect sub-pattern is used to describe a relatively specific defect description, which is a state between a software defect pattern and a software defect. It usually contains multiple specific software defects. The relationship between a software defect sub-pattern and a software defect pattern is that SDSM j ∈ SDM, j = 1, 2, …. The classification of software defect sub-patterns is shown in Table 3.
[0081] Table 3 Defect Sub-pattern Classification
[0082]
[0083]
[0084] According to the software defect model established by a, for step b, a perceptron is a mathematical model that mimics the neurons of organisms in real life. In biology, a neuron, that is, a neuron cell, is the most basic structural and functional unit of the nervous system and has the function of connecting and integrating input information and transmitting information. Neuron processes include dendrites and axons. Dendrites are numerous and their function is to receive impulses from the axons of other neurons and transmit them to the cell body. Axons are few in number and their function is to receive external stimuli. The synapses of the axons of neurons will connect to other neurons. Neurons generally have an active state and an inactive state. When in the active state, they can emit electrical pulses and transmit them to other neurons along two types of synapses.
[0085] The method includes:
[0086] B1 As Figure 3 shown, construct a DP-ANN neural network model, determine the activation function of the DP-ANN model. Here, the activation function is set to the softsign function, or it can also be other activation functions:
[0087]
[0088] B2 constructs the input layer, hidden layer, and output layer of the corresponding BP neural network according to the software defect model, where the input layer is the software defect pattern, the hidden layer is the software defect sub-pattern, and the output layer is the software test ability evaluation. For the convenience of representation, the input layer is denoted as the first layer, with a total of n perceptrons, and the output layer is denoted as output, which is the target layer. Assume that the model has several layers, and let be the value input from the i-th sensor of the (l - 1)-th layer, be the weight of the j-th sensor of the l-th layer for the input from the i-th sensor of the (l - 1)-th layer, be the output value after transformation by the activation function from the j-th sensor of the l-th layer, a l be the number of sensors in the l-th layer, be the bias of the j-th sensor of the l-th layer for all inputs from the previous layer; the summation output of the j-th sensor of the l-th layer is
[0089]
[0090] Then
[0091] B3 The actual j-th result obtained through training is compared with the expected result Then the formula for setting the mean square error function is:
[0092]
[0093] B4 In order to obtain an objective and reasonable weight, adjustments need to be made according to the error obtained in B3. The commonly used gradient descent method can be used to continuously reduce the loss function and update the weight in reverse. In order to accelerate the gradient descent, the idea of dynamic programming is introduced. Taking the calculation from the hidden layer to the output layer as an example (the principle from the input layer to the hidden layer is the same), it is characterized in that the learning rate η is determined, and η can be changed again for each calculation. Specifically as follows:
[0094] C1 Based on 3, the update of the parameters is as follows
[0095]
[0096] C2 Then according to (Equation 3), the partial derivative of can be obtained:
[0097]
[0098] According to in B2, taking the partial derivative of with respect to, the following can be obtained:
[0099]
[0100] C3 Order for but
[0101] C4 Since the software defect model established by a is applicable to the BP neural network of ANN, in addition, according to the software defect model, the correlation between each layer can be analyzed, which is very suitable for cutting the problem into several small problems. After solving them, they are strung together through their correlation to form the answer to the original problem. According to the idea of dynamic programming (DP), a problem is decomposed into several sub-problems, and the optimal solution is found for each sub-problem. The optimal solution of the previous sub-problem is provided to the next sub-problem as a basis, and each sub-problem is solved step by step. Finally, the last sub-problem is the optimal solution to the original problem.
[0102]
[0103] because is the output value of the jth perceptron in its layer after nonlinear transformation by the activation function softsign(x). Taking the derivative we get:
[0104]
[0105] The DP-ANN mathematical model finally established by C5 is:
[0106]
[0107] c. Here, the initialization and reset software defect sub-patterns are used as sub-problem solutions as examples. According to the constructed software defect pattern, the number of software defects at different detection stages and different hazard levels at the corresponding scale is used as the feature input, and the software defect hazard level is marked as 1, 2, and 3 according to low, medium, and high; static analysis, code review, and dynamic testing at the detection stage are marked as 1, 2, and 3; the declaration definition class and initialization class of the defect sub-pattern are marked as 0 and 1 respectively, and the test capability is divided into excellent and good as output, represented by 1 and 0 respectively, for testing; different software defects in the same software defect sub-pattern need to be summarized. The model is obtained through training data, and the test data obtained by different evaluation agencies are simulated to obtain the software testing capability results.
[0108] According to the preprocessed data, it is used as training input data, as shown in Table 4.
[0109] Table 4 Training input data examples
[0110]
[0111] After reading the training input data and output data, after completing the training, set the prediction input data, that is, the evaluation results of three evaluation units, and obtain the following data matrix:
[0112]
[0113] Set the prediction output data to obtain the following data matrix: 0 1 0
[0115] The number of nodes in the hidden layer is 2 because the initialization and reset classes are divided into initialization classes and declaration and definition classes. Through the established evaluation model based on DP-ANN, preprocess the data and train to obtain the optimal weights and thresholds required for the evaluation model. The results are as Figure 4 shown, the mean squared error is as Figure 5 shown, and the training status is as Figure 6 shown.
[0116] d. Apply the final calculation result to the software test ability evaluation of software defect patterns. The actually obtained results are 0, 1, 0; that is, the first and the third are good, and the second is excellent, as Figure 4 shown.
[0117] The mean absolute error MAE of the operation is: 0.16663, the mean squared error MSE is: 0.08326, and the root mean squared error RMSE is: 0.28855. It can be concluded that the training result is good. By solving this sub-problem, further expand the data to the entire model to evaluate the software test ability.
[0118] The present invention proposes a software test ability evaluation method based on DP-ANN and software defect patterns, which is applicable to the calculation of the weights of the evaluation system of software defect patterns, improves the efficiency of large-scale complex software defect evaluation work, and increases the objectivity of the weights.
[0119] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the technical principle of the present invention, several improvements and deformations can be made, and these improvements and deformations should also be regarded as the protection scope of the present invention.
Claims
1. A software test ability evaluation method based on DP-ANN and software defect patterns, characterized in that The method includes the following steps: S1. Preprocess various types of defect data of the software defect model, and construct a software defect pattern; S2. Establish a software test ability evaluation DP-ANN model for the software defect pattern based on the BP neural network algorithm of backpropagation, and optimize the gradient calculation of backpropagation; S3. Using the preprocessed data as input, through the established DP-ANN model, set the association relationship of each layer and the data volume of each layer, and train to obtain the optimal weights and thresholds required for the evaluation model; S4. Apply the final training result to the software test ability evaluation of the software defect pattern, use the optimal weights and thresholds obtained by training to form an evaluation model, input the result obtained from the actual test into the trained evaluation model to obtain an evaluation value, compare it with the actual observed value, and determine the evaluation result level according to the final output value, and conduct result analysis and discussion.
2. The software test ability evaluation method based on DP-ANN and software defect patterns according to claim 1, wherein, In S1, the data is divided according to the defect types, preprocessed by the principal component analysis and clustering analysis method, and a software defect pattern is constructed.
3. The software testing capability evaluation method based on DP-ANN and software defect patterns according to claim 2, wherein S1 includes: S11. Denote the software defect mode as SDM, the software defect sub-mode as SDSM, the software defect as SD, and the i-th software defect as SD i , and the j-th software defect sub-mode as SDSM j ; S12. Uniformly classify software defects. If there are J sub - patterns in total, and the j - th software defect sub - pattern has k types of software defects, the relationship is SD i ∈SDSM j , i = 1, 2, …, k, where SD i does not belong to sub - patterns other than SDSM j ; The relationship between software defect sub - patterns and software defect patterns is that SDSM j ∈SDM, j = 1, 2, … J.
4. The software testing capability evaluation method based on DP-ANN and software defect patterns according to claim 3, wherein, The software defect patterns include: initialization and reset class, calculation and algorithm class, logic design class, data processing class, interrupt timing design class, bus communication class, security design class, memory-related class, programming language specification class. Classify and manage different software defect patterns, and divide them into mandatory, optional, and conditional selection according to the software characteristics, and make explanations.
5. The software test ability evaluation method based on DP-ANN and software defect patterns according to claim 3 or 4, characterized in that S2 includes: S21. Determine the activation function of the DP-ANN model; S22. Construct the input layer, hidden layer, and output layer of the corresponding BP neural network according to the software defect model, where the input layer is the software defect pattern, the hidden layer is the software defect sub-pattern, and the output layer is the software test ability evaluation; denote the input layer as as the first layer, with a total of n perceptrons, and denote the output layer as output as the target layer. Assume that the model has several layers, and let be the value input from the i-th sensor in the (l - 1)-th layer, be the weight of the j-th sensor in the l-th layer for the input from the i-th sensor in the (l - 1)-th layer, be the output value after transformation by the activation function from the j-th sensor in the l-th layer, a l be the number of sensors in the l-th layer, be the bias of the j-th sensor in the l-th layer for all inputs from the previous layer; the summation output of the j-th sensor in the l-th layer is Then The actual j-th result obtained through training and the expected result are compared, and the formula for setting the mean square error function is as follows: S24. Make adjustments according to the error obtained in S23. According to the gradient descent method, continuously reduce the loss function, and update the weights backward. At the same time, in order to accelerate the gradient descent, introduce the dynamic programming DP idea, and finally establish the DP-ANN model.
6. The software testing ability evaluation method based on DP-ANN and software defect patterns according to claim 5, characterized in that In S21, the activation function is set to the softsign function:
7. The software testing ability evaluation method based on DP-ANN and software defect patterns according to claim 6, wherein S24 specifically includes: S241. Based on S23, the update of the parameters is as follows S242, then the partial derivative with respect to can be obtained according to formula (4): According to in S22, partial derivative is taken to obtain: S243. Let be then S244. For the software defect model, according to the idea of dynamic programming DP, decompose a problem into several sub-problems, find the optimal solution for each sub-problem, and use the optimal solution of the previous sub-problem as the basis for the next sub-problem, and gradually solve each sub-problem. Finally, the last sub-problem is the optimal solution of the original problem: because is the output value of the jth perceptron in its layer after nonlinear transformation by the activation function softsign(x). The derivative is: The finally established DP-ANN model is:
8. The software testing ability evaluation method based on DP-ANN and software defect patterns according to claim 7, characterized in that, The final output is divided into excellent and good.
9. The software test capability evaluation method based on DP-ANN and software defect patterns according to claim 7, characterized in that, In S4, according to the constructed software defect pattern, under the corresponding software scale, different software defect ratios and quantities under different detection stages and different hazard levels are used as features, and these data are used as training data to train an evaluation model. When different evaluation agencies participate in a project, for the test results of each evaluation agency, as prediction inputs, combined with the trained evaluation model, excellent and good evaluation ability results are obtained.
10. The software test capability evaluation method based on DP-ANN and software defect patterns according to claim 7, wherein, The S4 also includes: the mean absolute error (MAE), mean square error (MSE), and root mean square error (RMSE) obtained through training as indicators for evaluating the difference between the evaluation values of the DP-ANN model and the actual observed values, and are used to judge the fitting degree of the evaluation model on the given data.