Software testing capability evaluation method based on DP-ANN and software defect mode
Through the evaluation method based on DP-ANN and software defect model, the subjectivity and data sensitivity problems in large-scale complex software testing capabilities evaluation are solved, the evaluation efficiency and objectivity are improved, and a more fair software testing capabilities evaluation is achieved.
Patent Information
- Application Number
- CN202411088088.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2024-11-22
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
When evaluating the testing capabilities of large-scale complex software, the hierarchical analysis method is highly subjective, and the data envelope analysis method is sensitive to input and output data selection, making it difficult to apply to large data sets, resulting in low evaluation efficiency and poor objectivity.
Using the evaluation method based on DP-ANN and software defect mode, the backpropagation BP neural network model is established by preprocessing software defect data, the gradient calculation is optimized, the optimal weight and threshold are trained, and the evaluation model is formed, which is used for software testing. Ability assessment.
It improves the efficiency of large-scale complex software defect assessment and the objectivity of evaluation results, and enhances the fairness and effectiveness of the evaluation system.
Smart Images

Figure CN119003355A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of software testing, and in particular relates to a software testing capability evaluation method based on DP-ANN and software defect patterns. Background Art
[0002] With the continuous progress of the software industry, the scale of software is getting larger and larger, and the complexity is getting higher and higher. In the face of large-scale and highly complex software, the amount of software testing work is also increasing. The larger the scale of the software, the more problems it will have to test. Testing is a measure of software quality. Even in the face of highly complex software, the quality assessment of software testing work is usually conducted by examining the types and number of test problems.
[0003] In order to scientifically, efficiently and impartially evaluate the software testing capabilities of large-scale, highly complex software, it is usually necessary to quantify the results of software testing with a suitable model. In general, software problems are simply divided by category, number, and severity of the problems. This is a simple and effective way for small-scale, low-complexity software. In large and complex software, it is necessary to build an appropriate and good model layering of software defect patterns and solve them by dividing software defect patterns.
[0004] Based on large-scale and complex software, after building the software defect model, when evaluating the software testing capability, among most evaluation methods, the hierarchical analysis method is more subjective. When the software scale is small, it is simple, convenient and effective. When encountering such software, the increasing levels in the hierarchical analysis method will undoubtedly increase the work in various aspects. In addition, there are many indicators, and the weight of each indicator needs to be calculated separately. Moreover, the deeper the level of division, the less the advantages of the hierarchical analysis method can be reflected; using the data envelopment analysis method (DEA, Data Envelopment Analysis), determine each decision making unit (Decision Making Unit, DMU), and each decision making unit has the same input quantity and the same output quantity. In this case, determine the input and output, and how to proceed needs to be carefully considered. No matter how to determine, the final input and output must be proportional to the scale. The DEA method is very sensitive to the selection and definition of input and output data, and may be difficult to apply for large data sets and complex situations. Summary of the invention
[0005] 1. Technical issues to be resolved
[0006] The technical problem to be solved by the present invention is how to provide a software testing capability evaluation method based on DP-ANN and software defect patterns to solve the problem of software testing capability evaluation of large-scale and complex software.
[0007] (II) Technical solution
[0008] In order to solve the above technical problems, the present invention proposes a software testing capability evaluation method based on DP-ANN and software defect patterns, the method comprising the following steps:
[0009] S1. Preprocess various types of defect data of software defect models and construct software defect models;
[0010] S2. Establish a DP-ANN model for evaluating software testing capability of software defect patterns based on the back-propagation BP neural network algorithm, and optimize the gradient calculation of back-propagation;
[0011] S3. Based on the preprocessed data, as input, the established DP-ANN model is used to set the association relationship of each layer and the amount of data of each layer, and train to obtain the optimal weights and thresholds required for the evaluation model;
[0012] S4. Apply the final training results to the software testing capability assessment of software defect patterns. Use the optimal weights and thresholds obtained through training to form an assessment model. Input the results obtained through actual testing into the trained assessment model to obtain an assessment value. Compare it with the actual observed value. According to the final output value, establish the assessment result level, and conduct result analysis and discussion.
[0013] (III) Beneficial effects
[0014] The present invention proposes a software testing capability evaluation method based on DP-ANN and software defect patterns. The present invention proposes a software testing capability evaluation method based on DP-ANN and software defect patterns, which is suitable for calculating the weights of an evaluation system for software defect patterns, can improve the efficiency of large-scale complex software defect evaluation work, and increase the objectivity of the weights. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 It is a flow chart of the software testing capability evaluation method based on DP-ANN and software defect mode of the present invention;
[0016] Figure 2 It is a schematic diagram of software defect model;
[0017] Figure 3 Schematic diagram of DP-ANN model;
[0018] Figure 4 This is the running result diagram of initialization and reset software defects;
[0019] Figure 5 is the mean square error graph;
[0020] Figure 6 This is the training state diagram. DETAILED DESCRIPTION
[0021] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the drawings and examples.
[0022] The present invention aims to solve the above problems and improve the fairness, justice and effectiveness of the evaluation system. The specific implementation scheme is as follows:
[0023] The present invention provides a software testing capability evaluation method based on DP-ANN and software defect patterns, the method comprising the following steps:
[0024] S1. Preprocess various types of defect data of software defect models and construct software defect models;
[0025] In step S1, the data is divided according to the defect types, and preprocessed by principal component analysis, cluster analysis, etc. to construct a software defect model, which specifically includes the following steps:
[0026] S11. Software Defect Mode is SDM, Software Defect Sub-Mode is SDSM, Software Defect is SD, and the i-th software defect is SD i , the jth software defect sub-pattern is recorded as SDSM j .
[0027] S12. Classify the software defects uniformly. If there are J seed patterns in total, and the jth software defect sub-pattern has k types of software defects, then the relationship is SD i ∈SDSM j , i = 1, 2, ..., k, where SD i Not part of SDSM j Examples of sub-patterns other than are shown in Table 1.
[0028] Table 1 Examples of software defects
[0029]
[0030]
[0031] Classify and manage software defect modes such as initialization and reset, calculation and algorithm, logic design, data processing, interrupt timing design, bus communication, security design, memory-related, and programming language specification. According to the software characteristics, they are divided into mandatory, optional, and conditional, and explained, as shown in Table 2.
[0032] Table 2 Defect mode classification
[0033]
[0034] Defect sub-pattern is used to describe a relatively specific defect description, which is a statement between software defect pattern and software defect. It usually contains multiple specific software defects. The relationship between software defect sub-pattern and software defect pattern is: SDSM j ∈SDM, j = 1, 2, ... J. The classification of software defect sub-patterns is shown in Table 3.
[0035] Table 3 Defect sub-pattern classification
[0036]
[0037]
[0038] S2. Establish a DP-ANN model for evaluating software testing capability of software defect patterns based on the back-propagation BP neural network algorithm, and optimize the gradient calculation of back-propagation;
[0039] In the step S2, in the ANN model, the sensor is a mathematical model that imitates the neurons of biological organisms in real life. In biology, neurons, i.e., neuronal cells, are the most basic structural and functional units of the nervous system, and have the function of connecting and integrating input information and transmitting information. Neuronal protrusions include dendrites and axons. The number of dendrites is large, and their function is to receive impulses from the axons of other neurons and transmit them to the cell body. The number of axons is small, and their function is to receive external stimuli. The synapses of the axons of neurons will be connected to other neurons. Neurons generally have an activated state and an inactivated state. When in an activated state, they can send out electrical pulses, which are transmitted to other neurons along two synapses.
[0040] This step includes:
[0041] S21. Determine the activation function of the DP-ANN model. Here, the activation function is set to the softsign function, or other activation functions:
[0042]
[0043] S22. According to the software defect model, the input layer, hidden layer and output layer of the corresponding BP neural network are constructed, wherein the input layer is the software defect pattern, the hidden layer is the software defect sub-pattern, and the output layer is the software testing capability evaluation. For the convenience of representation, the input layer is recorded as is the first layer, with a total of n perceptrons, and the output layer is recorded as output as the target layer. Assume that the model has a total of several layers. is the input value from the i-th sensor in the l-1th layer, is the weight of the input from the jth sensor in the lth layer to the ith sensor in the l-1th layer, is the output value of the jth sensor from the lth layer after the activation function transformation, a l is the number of sensors in the lth layer, is the bias of the jth sensor in the lth layer for all inputs in the previous layer; the summed output of the jth sensor in the lth layer is
[0044]
[0045] but
[0046] S23, the actual j-th result obtained through training With expected results For comparison, the formula for setting the mean square error function is:
[0047]
[0048] S24, in order to obtain an objective and reasonable weight, it is necessary to make adjustments based on the error obtained in S23. The commonly used gradient descent method can be used to continuously reduce the loss function and update the weights in reverse. In order to speed up the gradient descent, the idea of dynamic programming is introduced. Taking the calculation from the hidden layer to the output layer as an example (the principle from the input layer to the hidden layer is the same), the learning rate η is determined. Each calculation can be replaced with a new η, and finally the DP-ANN model is established. The details are as follows:
[0049] S241. Based on S23, the update of parameters is as follows
[0050]
[0051] S242, then according to formula (4) we can get The partial derivative of :
[0052]
[0053] According to S22 right Taking partial derivatives, we can get:
[0054]
[0055] S243, Order for but
[0056] S244. Since the software defect model established in S1 is applicable to the BP neural network of ANN, in addition, according to the software defect model, the correlation between each layer can be analyzed, which is very suitable for cutting the problem into several small problems. After solving them, they are strung together through their correlation to form the answer to the original problem. For the software defect model, according to the idea of dynamic programming (DP), a problem is decomposed into several sub-problems, and the optimal solution is found for each sub-problem. The optimal solution of the previous sub-problem is provided to the next sub-problem as a basis, and each sub-problem is solved step by step. Finally, the last sub-problem is the optimal solution to the original problem.
[0057]
[0058] because is the output value of the jth perceptron in its layer after nonlinear transformation by the activation function softsign(x). Taking the derivative we get:
[0059]
[0060] S245. The DP-ANN model finally established is:
[0061]
[0062] S3. Based on the preprocessed data, as input, through the established DP-ANN model, set the association relationship of each layer, the amount of data of each layer, and train to obtain the optimal weights and thresholds required for the evaluation model.
[0063] S4. Apply the final training results to the software testing capability assessment of software defect patterns. Use the optimal weights and thresholds obtained through training to form an assessment model. The results obtained through actual testing are input into the trained assessment model to obtain an assessment value, which is compared with the actual observed value. According to the final output value, the assessment result level is established, and the results are analyzed and discussed. The mean absolute error MAE, mean square error MSE, root mean square error RMSE, etc. obtained through training are used as indicators to judge the difference between the evaluation value of the DP-ANN model and the actual observed value, and are used to judge the degree of fit of the assessment model on the given data.
[0064] Here, the final output is divided into excellent and good. According to the constructed software defect mode, the proportion and number of software defects at different detection stages and different hazard levels under the corresponding software scale are used as features, and these data are used as training data to train the evaluation model. When different evaluation agencies participate in a project, the test results of each evaluation agency are used as prediction input, combined with the trained evaluation model, to obtain excellent and good evaluation ability results.
[0065] Embodiment 1:
[0066] The following describes a specific implementation method in detail with reference to the accompanying drawings. To simplify the data scale, based on the idea of dynamic programming, initialization and resetting of software defect sub-patterns are used as sub-problems to solve the problem as an example.
[0067] A software testing capability evaluation method based on DP-ANN and software defect mode, the method comprises the following steps: Figure 1 :
[0068] a. Preprocess various types of defect data of software defect model and construct software defect model, see Figure 2 ;
[0069] b. Establish a software testing capability evaluation model for software defect patterns based on the back-propagation neural network algorithm and optimize the gradient calculation of back-propagation;
[0070] For step a, the data is divided according to the defect types, and preprocessed by principal component analysis, cluster analysis, etc. to construct a software defect model, which is characterized by specifically including the following steps:
[0071] A1 denotes the software defect mode (Software DefectMode) as SDM, the software defect sub-mode as SDSM (Software Defect Sub-Mode), the software defect (Software Defect) as SD, and the i-th software defect as SD i , the jth software defect sub-pattern is recorded as SDSM j .
[0072] A2 classifies software defects uniformly. If there are n sub-patterns in total and the j-th software defect sub-pattern has k types of software defects, then the relationship is SD i ∈SDSM j , i = 1, 2, ..., k, where SD i Not part of SDSM j Other sub-modes.
[0073] Table 1 Examples of software defects
[0074]
[0075]
[0076] Classify and manage software defect modes such as initialization and reset, calculation and algorithm, logic design, data processing, interrupt timing design, bus communication, security design, memory-related, and programming language specification. According to the software characteristics, they are divided into mandatory, optional, and conditional, and explained, as shown in Table 2.
[0077] Table 2 Defect mode classification
[0078]
[0079]
[0080] Defect sub-pattern is used to describe a relatively specific defect description, which is a statement between software defect pattern and software defect. It usually contains multiple specific software defects. The relationship between software defect sub-pattern and software defect pattern is: SDSM j ∈SDM, j = 1, 2, ... The classification of software defect sub-patterns is shown in Table 3.
[0081] Table 3 Defect sub-pattern classification
[0082]
[0083]
[0084] According to the software defect model established in a, for step b, the sensor is a mathematical model that imitates the neurons of real-life organisms. In biology, neurons are neuronal cells, which are the most basic structural and functional units of the nervous system. They have the function of connecting and integrating input information and transmitting information. Neuronal processes include dendrites and axons. The number of dendrites is large, and their function is to receive impulses from the axons of other neurons and transmit them to the cell body. The number of axons is small, and their function is to receive external stimuli. The synapses of the axons of neurons will connect with other neurons. Neurons generally have an activated state and an inactivated state. When in an activated state, they can send out electrical pulses and transmit them to other neurons along two synapses.
[0085] The method includes:
[0086] B1 Figure 3 As shown, a DP-ANN neural network model is constructed and the activation function of the DP-ANN model is determined. Here, the activation function is set to the softsign function, or other activation functions:
[0087]
[0088] B2 constructs the input layer, hidden layer and output layer of the corresponding BP neural network according to the software defect model, where the input layer is the software defect pattern, the hidden layer is the software defect sub-pattern, and the output layer is the software testing capability evaluation. For the convenience of representation, the input layer is recorded as is the first layer, with a total of n perceptrons, and the output layer is recorded as output as the target layer. Assume that the model has a total of several layers. is the input value from the i-th sensor in the l-1th layer, is the weight of the j-th sensor in the l-th layer for the input from the i-th sensor in the l-1-th layer, is the output value of the jth sensor from the lth layer after the activation function transformation, a l is the number of sensors in the lth layer, is the bias of the jth sensor in the lth layer for all inputs in the previous layer; the summed output of the jth sensor in the lth layer is
[0089]
[0090] but
[0091] B3 The actual j-th result obtained through training With expected results For comparison, the formula for setting the mean square error function is:
[0092]
[0093] In order to obtain an objective and reasonable weight, B4 needs to make adjustments based on the error obtained in B3. The commonly used gradient descent method can be used to continuously reduce the loss function and update the weights in reverse. In order to speed up the gradient descent, the idea of dynamic programming is introduced. Taking the calculation from the hidden layer to the output layer as an example (the principle from the input layer to the hidden layer is the same), the characteristic is that the learning rate η is determined, and η can be changed every time the calculation is performed, as follows:
[0094] C1 Based on 3, the update of parameters is as follows
[0095]
[0096] C2 can be obtained according to (Formula 3) The partial derivative of :
[0097]
[0098] According to B2 right Taking partial derivatives, we can get:
[0099]
[0100] C3 Order for but
[0101] C4 Since the software defect model established by a is applicable to the BP neural network of ANN, in addition, according to the software defect model, the correlation between each layer can be analyzed, which is very suitable for cutting the problem into several small problems. After solving them, they are strung together through their correlation to form the answer to the original problem. According to the idea of dynamic programming (DP), a problem is decomposed into several sub-problems, and the optimal solution is found for each sub-problem. The optimal solution of the previous sub-problem is provided to the next sub-problem as a basis, and each sub-problem is solved step by step. Finally, the last sub-problem is the optimal solution to the original problem.
[0102]
[0103] because is the output value of the jth perceptron in its layer after nonlinear transformation by the activation function softsign(x). Taking the derivative we get:
[0104]
[0105] The DP-ANN mathematical model finally established by C5 is:
[0106]
[0107] c. Here, the initialization and reset software defect sub-patterns are used as sub-problem solutions as examples. According to the constructed software defect pattern, the number of software defects at different detection stages and different hazard levels at the corresponding scale is used as the feature input, and the software defect hazard level is marked as 1, 2, and 3 according to low, medium, and high; static analysis, code review, and dynamic testing at the detection stage are marked as 1, 2, and 3; the declaration definition class and initialization class of the defect sub-pattern are marked as 0 and 1 respectively, and the test capability is divided into excellent and good as output, represented by 1 and 0 respectively, for testing; different software defects in the same software defect sub-pattern need to be summarized. The model is obtained through training data, and the test data obtained by different evaluation agencies are simulated to obtain the software testing capability results.
[0108] According to the preprocessed data, it is used as training input data, as shown in Table 4.
[0109] Table 4 Training input data examples
[0110]
[0111] After reading the training input data and output data, and completing the training, set the prediction input data, that is, the evaluation results of the three evaluation units, and obtain the following data matrix:
[0112]
[0113] Set the predicted output data to get the following data matrix: 0 1 0
[0115] The number of nodes in the hidden layer is 2 because the initialization and reset classes are divided into initialization classes and declaration definition classes. By establishing an evaluation model based on DP-ANN, preprocessing data, and training, the optimal weights and thresholds required for the evaluation model are obtained. The results are as follows Figure 4 As shown, the mean square error is Figure 5 As shown, the training status is Figure 6 shown.
[0116] d. Apply the final calculation results to the software testing capability evaluation of software defect patterns. The actual results are 0, 1, 0; that is, the first and third companies are good, and the second company is excellent. Figure 4 shown.
[0117] The mean absolute error (MAE) of the operation is 0.16663, the mean square error (MSE) is 0.08326, and the root mean square error (RMSE) is 0.28855, which shows that the training results are good. By solving this sub-problem, the data is further expanded to the entire model to evaluate the software testing capabilities.
[0118] The present invention proposes a software testing capability evaluation method based on DP-ANN and software defect patterns, which is suitable for calculating the evaluation system weights of software defect patterns, improving the efficiency of large-scale complex software defect evaluation work, and increasing the objectivity of weights.
[0119] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A software testing capability evaluation method based on DP-ANN and software defect patterns, characterized in that: The method comprises the following steps: S1. Preprocess various types of defect data of software defect models and construct software defect models; S2. Establish a DP-ANN model for evaluating software testing capability of software defect patterns based on the back-propagation BP neural network algorithm, and optimize the gradient calculation of back-propagation; S3. Based on the preprocessed data, as input, the established DP-ANN model is used to set the association relationship of each layer and the amount of data of each layer, and train to obtain the optimal weights and thresholds required for the evaluation model; S4. Apply the final training results to the software testing capability assessment of software defect patterns. Use the optimal weights and thresholds obtained through training to form an assessment model. Input the results obtained through actual testing into the trained assessment model to obtain an assessment value. Compare it with the actual observed value. According to the final output value, establish the assessment result level, and conduct result analysis and discussion.
2. The software testing capability evaluation method based on DP-ANN and software defect pattern according to claim 1, characterized in that: In S1, the data is divided according to the defect types, pre-processed by principal component analysis and cluster analysis to construct a software defect model.
3. The software testing capability evaluation method based on DP-ANN and software defect pattern according to claim 2, characterized in that: The S1 includes: S11. The software defect mode is SDM, the software defect sub-mode is SDSM, the software defect is SD, and the i-th software defect is SD i , the jth software defect sub-pattern is recorded as SDSM j ; S 12. Classify the software defects uniformly. If there are J seed patterns in total, and the j-th software defect sub-pattern has k types of software defects, then the relationship is SD i ∈SDSM j , i = 1, 2, ..., k, where SD i Not part of SDSM j The relationship between the software defect sub-pattern and the software defect pattern is: SDSM j ∈SDM, j=1,2,...J.
4. The software testing capability evaluation method based on DP-ANN and software defect pattern according to claim 3, characterized in that: Software defect modes include: initialization and reset class, calculation and algorithm class, logic design class, data processing class, interrupt timing design class, bus communication class, security design class, memory-related class, and programming language specification class. Different software defect modes are classified and managed, and divided into mandatory, optional, and conditional selection according to software characteristics, and explained.
5. The software testing capability evaluation method based on DP-ANN and software defect pattern according to claim 3 or 4, characterized in that: The S2 includes: S21, determining the activation function of the DP-ANN model; S22. Construct the input layer, hidden layer and output layer of the corresponding BP neural network according to the software defect model, wherein the input layer is the software defect pattern, the hidden layer is the software defect sub-pattern, and the output layer is the software testing capability evaluation; the input layer is denoted as is the first layer, with a total of n perceptrons, and the output layer is recorded as output as the target layer. Assume that the model has a total of several layers. is the input value from the i-th sensor in the l-1th layer, is the weight of the input from the jth sensor in the lth layer to the ith sensor in the l-1th layer, is the output value of the jth sensor from the lth layer after the activation function transformation, a l is the number of sensors in the lth layer, is the bias of the jth sensor in the lth layer for all inputs in the previous layer; the summed output of the jth sensor in the lth layer is but S23, the actual j-th result obtained through training With expected results For comparison, the formula for setting the mean square error function is: S24. Make adjustments based on the error obtained in S23. According to the gradient descent method, make the loss function smaller and update the weights in reverse. At the same time, in order to speed up the gradient descent, introduce the dynamic programming DP idea and finally establish the DP-ANN model.
6. The software testing capability evaluation method based on DP-ANN and software defect pattern according to claim 5, characterized in that: In S21, the activation function is set to a softsign function:
7. The software testing capability evaluation method based on DP-ANN and software defect pattern according to claim 6, characterized in that: The S24 specifically includes: S241. Based on S23, the update of parameters is as follows S242, then according to formula (4) we can get The partial derivative of : According to S22 right Taking partial derivatives, we get: S243, Order for but S244. For the software defect model, according to the idea of dynamic programming (DP), a problem is decomposed into several sub-problems, and the optimal solution is found for each sub-problem. The optimal solution of the previous sub-problem is provided as a basis for the next sub-problem, and each sub-problem is solved step by step. Finally, the last sub-problem is the optimal solution to the original problem: because is the output value of the jth perceptron in its layer after nonlinear transformation by the activation function softsign(x). The derivative is: S245. The DP-ANN model finally established is:
8. The software testing capability evaluation method based on DP-ANN and software defect pattern according to claim 7, characterized in that: The final output is divided into excellent and good.
9. The software testing capability evaluation method based on DP-ANN and software defect pattern according to claim 7, characterized in that: In S4, the proportion and number of software defects at different detection stages and different hazard levels of the constructed software defect pattern are used as features under the corresponding software scale, and these data are used as training data to train an evaluation model. When different evaluation agencies participate in a project, the test results of each evaluation agency are used as prediction input and combined with the trained evaluation model to obtain excellent and good evaluation capability results.
10. The software testing capability evaluation method based on DP-ANN and software defect pattern according to claim 7, characterized in that: The S4 also includes: the mean absolute error MAE, mean square error MSE, and root mean square error RMSE obtained through training as indicators for judging the difference between the evaluation value of the DP-ANN model and the actual observation value, and for judging the degree of fit of the evaluation model on the given data.
