A software performance quality evaluation method and device and a storage medium

Abstract

The application relates to a software performance quality evaluation method and device and a storage medium, and the method comprises the following steps: data collection: collecting historical data reflecting software performance quality to form a sample set, each sample comprising a plurality of performance quality index data; performing principal component analysis based on the sample set to extract and refine a plurality of principal components, and determining a principal component expression; constructing a principal component comprehensive model for evaluating software performance quality based on the extracted principal components; and performing performance quality evaluation of the software to be evaluated according to the principal component comprehensive model. Compared with the prior art, the application can comprehensively and comprehensively evaluate the software performance quality, accurately reflect the performance of the software system in a period of time, and has high evaluation accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of software testing, in particular to a software performance quality evaluation method and device and storage medium. BACKGROUND

[0002] The existing software quality evaluation adopts an expert scoring method, sets weight values corresponding to various indexes, each expert scores various indexes of the software, then calculates the degree of excellence of the software indexes in combination with the scoring results and weight values of the various indexes, and finally calculates and presents the final evaluation result of the software quality according to the degree of excellence.

[0003] The main evaluation indexes include four first-level indexes of functionality, reliability, maintainability and portability, the second-level indexes corresponding to the functionality include accuracy, safety and compliance, the second-level indexes corresponding to the reliability include maturity, error tolerance and easy recovery, the second-level indexes of the maintainability include easy analysis, stability and easy testing, and the second-level indexes corresponding to the portability include compliance, installability and transferability. The software system evaluation grade and the indexes that need to be improved of the software are given in the final evaluation result, which gives a guiding suggestion for further optimization of the software.

[0004] The existing evaluation method mainly focuses on the comprehensive quality of the software, and the performance quality of the software is only used as one evaluation point, so the performance quality of the software cannot be comprehensively evaluated; the final evaluation result of the existing technology is not focused on the overall quality of the system, so it cannot accurately guide the performance testers to organize and carry out the performance test special work; the indexes selected by the existing technology are values at one time point, and there is no statistical analysis of the historical data of the software in a period of time, so the result obtained at the present time cannot be used to predict the performance of the software system in a period of time. SUMMARY

[0005] The present application provides a software performance quality evaluation method, device and storage medium to overcome the defects of the existing technology, so as to comprehensively evaluate the performance quality of the software and accurately reflect the performance of the software system in a period of time.

[0006] The object of the present application can be achieved by the following technical solutions.

[0007] A software performance quality evaluation method, the method comprising:

[0008] Data collection: collect historical data reflecting the performance quality of the software to form a sample set, each sample including a plurality of performance quality index data;

[0009] Perform principal component analysis based on the sample set to extract and refine a plurality of principal components, and determine a principal component expression;

[0010] A principal component synthesis model for evaluating software performance quality was constructed based on the extracted principal components.

[0011] The performance and quality of the software to be evaluated are assessed based on the principal component synthesis model.

[0012] Preferably, the performance quality indicators include a first type of indicator and a second type of indicator, wherein the second type of indicator is calculated from the first type of indicator.

[0013] Preferably, the first type of indicators includes the number of versions, the number of versions covered by load testing, the number of versions not tested, the project level, the number of UAT performance defects, and the number of production defects. The second type of indicators includes load testing coverage, production performance defect rate, performance missed test rate, and performance defect active identification rate.

[0014] Preferably, the calculation method for the two types of indicators includes:

[0015]

[0016]

[0017]

[0018] Preferably, the principal component analysis specifically includes:

[0019] Standardize the performance quality index data of all samples in the sample set and construct a sample set standardization matrix;

[0020] Using performance quality indicators as variables, a correlation coefficient matrix of the variables is established;

[0021] Find the eigenvalues ​​and corresponding unit eigenvectors of the correlation coefficient matrix, and calculate the principal component contribution rate and cumulative contribution rate.

[0022] Extract the principal components corresponding to the feature values ​​whose cumulative contribution rate exceeds a set size and construct the principal component expression.

[0023] Preferably, the principal component expression is represented as follows:

[0024] Y i =a i1 ·ZX1+a i2 ·ZX2+…+a ip ZX p

[0025]

[0026] Among them, Y i Let a be the expression for the i-th principal component, i = 1, 2, ..., I, where I is the total number of extracted principal components. iZX is a unit eigenvector corresponding to the eigenvalue of the i-th principal component, ZX j Xi represents the standard value of the j-th performance quality index data, j = 1, 2, …, p, and p is the total number of performance quality indexes.

[0027] Preferably, the principal component comprehensive model is represented as:

[0028]

[0029] Y is a comprehensive principal component value, λ i ZX is a unit eigenvector corresponding to the eigenvalue of the i-th principal component, ZX k ZX is a unit eigenvector corresponding to the eigenvalue of the k-th principal component.

[0030] Preferably, the performance quality evaluation of the software to be evaluated according to the principal component comprehensive model comprises:

[0031] calculating the comprehensive principal component value of the software to be evaluated by using the principal component comprehensive model;

[0032] The software performance quality is inversely proportional to the comprehensive principal component value, and the lower the comprehensive principal component value, the higher the software performance quality.

[0033] A software performance quality evaluation device based on principal component analysis, comprising a memory and a processor, the memory is used to store a computer program, and the processor is used to realize the software performance quality evaluation method when the computer program is executed.

[0034] A storage medium having a computer program stored thereon, the computer program being executed by a processor to realize the software performance quality evaluation method.

[0035] Compared with the prior art, the present application has the following advantages:

[0036] (1) The present application selects a more comprehensive performance quality related index, collects real historical data as the operation basis, can more reasonably evaluate the system comprehensive performance quality, the reference dimension and index are more comprehensive, and the performance quality is more guaranteed;

[0037] (2) All software systems of the present application adopt a unified standard output comprehensive quality, the reference dimension and index adopted are the industry concerned indexes of system quality, the data relied on is real historical data, and the quality model calculated based thereon has high reference value;

[0038] (3) The principal component analysis method adopted by the present application extracts a few comprehensive main components from the original indexes, makes them retain as much information of the original indexes as possible, and are not related to each other, can eliminate the mutual influence between the evaluation indexes, and makes the evaluation result accurate and reliable. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 A flowchart of a software performance quality evaluation method of the present application. DETAILED DESCRIPTION

[0040] The present application will be described in detail below with reference to the accompanying drawings and specific embodiments. Note that the following description of embodiments is merely illustrative and the present application is not intended to limit the scope of application or its use, and the present application is not limited to the following embodiments.

[0041] Embodiment 1

[0042] As shown in the following table, the present embodiment provides a software performance quality evaluation method, which comprises: Figure 1

[0043] Data collection: collect historical data reflecting software performance quality to form a sample set, each sample including a plurality of performance quality indicator data;

[0044] Based on the sample set, principal component analysis is performed to extract and refine a plurality of principal components, and a principal component expression is determined;

[0045] Based on the extracted principal components, a principal component comprehensive model for evaluating software performance quality is constructed;

[0046] According to the principal component comprehensive model, the performance quality of the software to be evaluated is evaluated.

[0047] The performance quality indicators include a type of indicators and a type of indicators, and the type of indicators is calculated based on the type of indicators. Specifically, the type of indicators includes version number, stress test coverage version number, non-stress test version number, project level, UAT performance defect number, and production defect number, and the type of indicators includes stress test coverage rate, production performance defect rate, performance leakage rate, and performance defect active identification rate. The calculation method of the type of indicators is as follows:

[0048]

[0049]

[0050]

[0051] It should be noted that for the case where the denominator is 0, the value is determined as 0 or 1 according to the actual business scenario.

[0052] In the present embodiment, after the various types of data generated by the 18 software projects participated in the implementation are sorted and summarized, the specific data of 6 type indicators (direct indicators) and 4 type indicators (indirect indicators) are obtained as shown in Table 1:

[0053] Table 1 Performance quality indicator data of software projects​

[0054]

[0055]

[0056] Principal component analysis (PCA) is a technique for simplifying a dataset in statistics. It extracts and refines a few comprehensive main components from the original indicators, enabling them to retain as much information as possible from the original indicators and being mutually independent. This technique aims at dimensionality reduction. Through complex linear transformations, the original data is transformed into a new coordinate system, such that the first largest variance of any data projection lies on the first coordinate (referred to as the first principal component), the second largest variance lies on the second coordinate (the second principal component), and so on.

[0057] The main derivation process is as follows:

[0058] Suppose there are n samples, and each sample has p variables, forming an n×p - order data matrix. When p is relatively large, it is troublesome to examine problems in the p - dimensional space. To overcome this difficulty, dimensionality reduction processing is required, that is, using a few comprehensive indicators to replace the original more variable indicators, and making these fewer comprehensive indicators be able to reflect as much information as possible from the original more variable indicators, while being independent of each other.

[0059] Definition: Denote x1, x2, …, x p as the original variable indicators, and z1, z2, …, z m (m < p) as the new variable indicators.

[0060]

[0061] l i1 2 +…+l ip 2 = 1

[0062] Among them:

[0063] 1. z i and z j are mutually independent;

[0064] 2. z1 is the one with the largest variance among all linear combinations of x1, x2, …, x p , and z2 is the one with the largest variance among all linear combinations of x1, x2, …, x p that is uncorrelated with z1; …; z m is the one with the largest variance among all linear combinations of x1, x2, …, x m (m < p) that are all uncorrelated with z1, z2, …, z pThe new variable indicators z1, z2, …, z m (m < p) are called the original variable indicators x1, x2, …, x p The first, second, …, and mth principal components.

[0065] Suppose there are n samples, each sample observing p indicators, and the original data is written as a matrix.

[0066]

[0067] a. Standardize the original data.

[0068]

[0069] b. Establish the correlation coefficient matrix of the variables

[0070]

[0071] R = (r ij ) p×p

[0072]

[0073] c. Find the eigenvalues of R and the corresponding unit eigenvectors

[0074]

[0075] F i = a 1i X1 + a 2i X2 + … + a pi X p , (i = 1, …, p), and write the principal components. Calculate the principal component contribution rate and cumulative contribution rate

[0076] Contribution rate:

[0077] Cumulative contribution rate:

[0078] Take the eigenvalues λ1, λ2, …, λ m corresponding to the first, second, …, and mth (m ≤ p) principal components.

[0079] In summary, the principal component analysis of the present embodiment specifically includes:

[0080] Standardize the performance quality indicator data of all samples in the sample set to construct a sample set standardized matrix.

[0081] Take the performance quality indicators as variables to establish a correlation coefficient matrix of the variables.

[0082] Eigenvalues and corresponding unit eigenvectors of the correlation coefficient matrix are calculated, and principal component contribution rate and cumulative contribution rate are calculated.

[0083] The principal components corresponding to the eigenvalues with cumulative contribution rate exceeding a set size are extracted to construct a principal component expression.

[0084] The principal component expression is represented as:

[0085] Y i = a i1 · ZX1 + a i2 · ZX2 + … + a ip · ZX p

[0086]

[0087] where Y i is the expression of the i-th principal component, i = 1, 2, …, I, I is the total number of extracted principal components, a i is the unit eigenvector corresponding to the i-th principal component, ZX j represents the standard value of the j-th performance quality indicator data, j = 1, 2, …, p, p is the total number of performance quality indicators.

[0088] The principal component comprehensive model is represented as:

[0089]

[0090] where Y is the comprehensive principal component value, λ i is the eigenvalue corresponding to the i-th principal component, λ k is the eigenvalue corresponding to the k-th principal component.

[0091] The performance quality evaluation of the software to be evaluated according to the principal component comprehensive model includes:

[0092] The comprehensive principal component value of the software to be evaluated is calculated using the principal component comprehensive model;

[0093] The software performance quality is inversely proportional to the comprehensive principal component value, and the lower the comprehensive principal component value, the higher the software performance quality.

[0094] The above describes the data processing and model establishment process in detail:

[0095] Step 1: Data standardization, as shown in Table 2:

[0096] Table 2 Standardization results of performance quality indicator data of software project

[0097]

[0098]

[0099] The standardized matrix is as follows:

[0100]

[0101] Second, the correlation coefficient matrix of variables is established, as shown in Table 3:

[0102] Table 3 Correlation coefficient matrix construction diagram

[0103]

[0104]

[0105] The correlation coefficient matrix is as follows:

[0106]

[0107] Third, the eigenvalue, contribution rate and cumulative contribution rate are shown in Table 4:

[0108] Table 4 Eigenvalue, contribution rate and cumulative contribution rate

[0109] Serial number Characteristic value λ Contribution rate % Cumulative contribution rate % 1 3.73 37.295 37.295 2 2.611 26.109 63.404 3 1.507 15.066 78.47 4 1.003 10.034 88.504

[0110] The starting eigenvalue greater than 1 is a general standard for useful factors. When the eigenvalue is less than 1, it means that the information obtained in this factor is not enough to prove that it should be retained.

[0111] The cumulative contribution rate is used to explain the contribution rate of the factor, and the higher the cumulative percentage indicates that the degree of explanation of these factors to the overall is higher. Generally, the cumulative percentage is higher than 70%, which indicates that it is satisfactory.

[0112] Fourth, the unit eigenvector is shown in Table 5:

[0113] Table 5 Unit eigenvector

[0114]

[0115] Therefore:

[0116] The eigenvector corresponding to the eigenvalue λ1=3.73 is as follows:

[0117]

[0118] The eigenvector corresponding to the eigenvalue λ2=2.611 is as follows:

[0119]

[0120] The eigenvector corresponding to the eigenvalue λ3=1.507 is as follows:

[0121]

[0122] The eigenvector corresponding to the eigenvalue λ4= 1.003 is as follows:

[0123]

[0124] Fourth step, principal component expression

[0125] The unit eigenvector Ui is multiplied by the standard value ZX of the 10 variables i to obtain the expression of the 4 principal components Y1, Y2, Y3, Y4:

[0126] Y1= 0.333 x ZX1- 0.370 x ZX2+ 0.440 x ZX3+... - 0.198 x ZX 10

[0127] Y2= -0.073 x ZX1+ 0.286 x ZX2- 0.200 x ZX3+... - 0.007 x ZX 10

[0128] Y3= 0.408 x ZX1+ 0.057 x ZX2+ 0.279 x ZX3+... + 0.670 x ZX 10

[0129] Y4= 0.369 x ZX1+ 0.128 x ZX2+ 0.214 x ZX3+... - 0.062 x ZX 10

[0130] Fifth step, comprehensive principal component value

[0131] The proportion of the eigenvalue corresponding to the 4 principal components in the sum of the eigenvalues of the extracted principal components is taken as the weight to calculate the principal component comprehensive model. According to the principal component comprehensive model, the comprehensive principal component value can be calculated, which is represented as:

[0132]

[0133] The comprehensive principal component values of the 18 projects obtained by calculation are shown in Table 6:

[0134] Table 6 Comprehensive principal component value table

[0135]

[0136]

[0137] Through the comprehensive operation of each index, each system will finally obtain a comprehensive score, and the current performance quality is ranked according to the final score, the lower the system score, the higher the system performance quality, and the lower the probability of production performance problem, thus a unified standard can be used to evaluate the performance quality of all internal systems, guide the daily performance test work, develop a reasonable performance inspection plan, and predict the risk of system performance problems in a period of time.

[0138] The application focuses on evaluating the performance quality of a software system, by selecting index data related to software performance, collecting real running data in a period of time, including test environment and production environment data as sample data, and obtaining the current performance quality result of the software system through scientific operation of the sample data. The proposal is based on real historical sample data of software, collects performance-related indicators, and the analysis result can effectively reflect the current performance of the software system, and serve as a reference standard for predicting the stability of the system performance in the future.

[0139] Embodiment 2

[0140] The embodiment provides a software performance quality evaluation device based on principal component analysis, including a memory and a processor, the memory is used for storing a computer program, and the processor is used for realizing the software performance quality evaluation method in embodiment 1 when the computer program is executed. The software performance quality evaluation method has been specifically described in embodiment 1, and will not be repeated here.

[0141] Embodiment 3

[0142] The embodiment provides a storage medium having a computer program stored thereon, and the computer program is executed by a processor to realize the software performance quality evaluation method in embodiment 1. The software performance quality evaluation method has been specifically described in embodiment 1, and will not be repeated here.

[0143] The above embodiments are only examples and do not limit the scope of the application. These embodiments can be implemented in various ways and can be omitted, replaced, or changed without departing from the technical idea of the application.

Claims

1. A software performance quality evaluation method, characterized in that, The method includes: Data acquisition: Collect historical data reflecting software performance and quality to form a sample set, with each sample including multiple performance and quality indicator data; Principal component analysis is performed on the sample set to extract and refine several principal components, and the principal component expressions are determined. A principal component synthesis model for evaluating software performance quality was constructed based on the extracted principal components. The performance and quality of the software to be evaluated are assessed based on the principal component synthesis model. The performance quality indicators include a first-class indicator and a second-class indicator, wherein the second-class indicator is calculated from the first-class indicator; The first type of indicators includes the number of versions, the number of versions covered by load testing, the number of versions not tested, the project level, the number of UAT performance defects, and the number of production defects. The second type of indicators includes load testing coverage, production performance defect rate, performance missed test rate, and performance defect active identification rate. The calculation methods for the two types of indicators include:

2. The software performance quality evaluation method according to claim 1, characterized in that, The principal component analysis specifically includes: Standardize the performance quality index data of all samples in the sample set and construct a sample set standardization matrix; Using performance quality indicators as variables, establish a correlation coefficient matrix for the variables; Find the eigenvalues ​​and corresponding unit eigenvectors of the correlation coefficient matrix, and calculate the principal component contribution rate and cumulative contribution rate. Extract the principal components corresponding to the feature values ​​whose cumulative contribution rate exceeds a set size and construct the principal component expression.

3. The software performance quality evaluation method according to claim 2, characterized in that, The principal component expression is as follows: Y i =a i1 ·ZX1+a i2 ·ZX2+…+a ip ·ZX p Among them, Y i Let a be the expression for the i-th principal component, i = 1, 2, ..., I, where I is the total number of extracted principal components. i ZX is the unit eigenvector corresponding to the eigenvalue of the i-th principal component. j Let j represent the standard value of the j-th performance quality index data, where j = 1, 2, ..., p, and p is the total number of performance quality indices.

4. The software performance quality evaluation method according to claim 3, characterized in that, The principal component synthesis model is expressed as follows: Where Y is the overall principal component value, λ i Let λ be the eigenvalue corresponding to the i-th principal component. k is the eigenvalue corresponding to the k-th principal component.

5. The software performance quality evaluation method according to claim 4, characterized in that, The performance and quality evaluation of the software to be evaluated based on the principal component synthesis model includes: The principal component values ​​of the software to be evaluated are calculated using the principal component synthesis model. Software performance quality is inversely proportional to the overall principal component score; the lower the overall principal component score, the higher the software performance quality.

6. A software performance quality evaluation device, characterized in that, It includes a memory and a processor, the memory being used to store a computer program, and the processor being used to implement the software performance quality assessment method as described in any one of claims 1 to 5 when the computer program is executed.

7. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the software performance quality evaluation method as described in any one of claims 1 to 5.

Images