Construction method of software reliability evaluation model

Abstract

The invention relates to a method for constructing a software reliability evaluation model, which comprises the following steps of: A, acquiring multi-source data, and preprocessing the multi-source data; b, constructing a multivariate smooth spline regression model; c, updating parameters of the multivariate smooth spline regression model in real time according to the prior distribution and operation data through a Bayesian formula; d, parameters of the multivariate smooth spline regression model are updated in real time through the Kalman filtering technology, and model parameter prediction, prediction error covariance calculation, Kalman gain calculation, state updating and error covariance updating are sequentially carried out every time operation data arrives; and E, training and verifying the updated multivariate smooth spline regression model to obtain a final evaluation model. According to the method, the evaluation precision and robustness of the software reliability are improved, the calculation efficiency and the automation degree are improved, and the result interpretability and the data utilization efficiency are enhanced.

Description

Technical Field

[0001] The present invention belongs to the field of machine learning algorithm model construction, and specifically relates to a method for constructing a software reliability assessment model. Background Art

[0002] In today's digital age, the reliability of software in safety-critical areas is of paramount importance to ensure the safe operation of the system. Therefore, accurate reliability assessment of software is also crucial. However, existing methods for assessing software reliability have many deficiencies, which hinder efficient and accurate reliability assessment. Existing reliability assessment methods have at least the following problems:

[0003] Single variable limitation: Traditional software reliability evaluation methods are usually based on a single variable or a few variables for modeling, and fail to fully consider the multiple factors that affect software reliability. This method ignores the complex relationship between multiple variables such as code complexity, test coverage, and usage environment, resulting in incomplete and inaccurate evaluation results.

[0004] Limitations of static models: Most existing reliability assessment models are static and cannot dynamically adjust model parameters based on the arrival of new data. This means that these models lack sufficient flexibility and adaptability in the face of ever-changing software development and operating environments, making it difficult to provide real-time reliability assessment and prediction.

[0005] Data fusion and processing challenges: As data sources in the software development process become increasingly diverse, how to effectively fuse and process multi-source data has become a major challenge. Existing methods often perform poorly in dealing with data inconsistency and noise, which affects the accuracy and robustness of the model.

[0006] Another example is traditional statistical analysis methods, such as nonparametric statistical models and survival analysis models, which can provide a certain degree of predictive ability under certain conditions, but they are often unable to cope with the complex nonlinear relationships and dynamic changes in software systems. These methods are usually based on historical data analysis and are difficult to capture the local characteristics and real-time changes of the system.

[0007] Although machine learning and deep learning-based methods can handle large-scale data and complex patterns, they often require a large amount of data for training, and the models are poorly interpretable, making it difficult to explain the causes and basis of the model results. In addition, these methods usually require complex computing resources and long training times, limiting their application in real-time evaluation. Although time series analysis methods are suitable for processing time series data, they are often limited to linear relationships and stationary data, and have limited processing capabilities for multidimensional data and nonlinear relationships. Summary of the invention

[0008] The present invention provides a method for constructing a software reliability evaluation model, which can dynamically evaluate the reliability of software according to various factors affecting the software reliability, thereby improving the accuracy of the evaluation and the robustness of the model.

[0009] The method for constructing a software reliability evaluation model of the present invention comprises the steps of:

[0010] A. Acquire multi-source data and pre-process the multi-source data;

[0011] B. Construct a multivariate smoothing spline regression model;

[0012] C. Update the parameters of the multivariate smooth spline regression model in real time based on the prior distribution and the operating data obtained in real time during the program operation through the Bayesian formula;

[0013] D. then updating the parameters of the multivariate smooth spline regression model in real time through the Kalman filtering technology, and performing parameter prediction, prediction error covariance calculation, Kalman gain calculation, state update and error covariance update of the multivariate smooth spline regression model in sequence each time the operation data arrives;

[0014] E. Train and validate the updated multivariate smooth spline regression model to obtain the final evaluation model.

[0015] The present invention solves the problem that traditional software reliability evaluation methods usually only model based on a single variable or a few variables, and fail to fully consider the multiple factors that affect software reliability. In addition, the present invention comprehensively considers the complex relationship between multiple variables such as software code complexity, test coverage, and usage environment, and ultimately greatly improves the comprehensiveness and accuracy of software reliability evaluation results.

[0016] Furthermore, the multi-source data in step A includes: development process data, test data, operation logs and user feedback data.

[0017] Furthermore, in step A, the preprocessing includes: missing value processing, outlier detection, cross-source data alignment, feature extraction, feature correlation analysis, feature dimensionality reduction processing, data normalization and encoding processing.

[0018] Furthermore, in step B, a smooth spline function is used to capture the nonlinear relationship between the features in the multi-source data and the target variable, and the final multivariate smooth spline regression model is obtained.

[0019] Furthermore, in step C, the Bayesian formula is used to calculate the posterior distribution through the prior distribution and the likelihood function, and the parameters of the multivariate smooth spline regression model are updated in real time through the posterior distribution.

[0020] Furthermore, in step D, the state transfer matrix and control matrix in the Kalman filter technology are first determined, and then the state variables and error covariance are initialized. Then, each time operation data arrives, the parameter prediction, prediction error covariance calculation, Kalman gain calculation, state update and error covariance update of the multivariate smooth spline regression model are performed in sequence.

[0021] Furthermore, in step E, when training the updated multivariate smooth spline regression model, the K-fold cross validation method is used to train the model for different software development life cycle stages, so that the model of each stage can accurately reflect the current reliability status; when verifying the trained multivariate smooth spline regression model, the K-fold cross validation method is used to evaluate the model performance to ensure the generalization ability and robustness of the model and obtain the final evaluation model.

[0022] Specifically, the multi-source data is randomly divided into k subsets of equal size, iterated k times, each time a different subset is selected as a validation set, and the remaining k-1 subsets are used as training sets; in each iteration, the training set is used to train the multivariate smooth spline regression model, and the performance of the multivariate smooth spline regression model is evaluated on the validation set, and the average and standard deviation of the performance indicators of the k validations are calculated as the final evaluation result of the model.

[0023] Furthermore, the method also includes step F: optimizing parameters of the evaluation model using a genetic algorithm.

[0024] Specifically, step F includes:

[0025] F1. Initialize the population: Initialize a group of randomly generated individuals, each of which consists of a set of hyperparameters;

[0026] F2. Fitness evaluation: Evaluate each individual and calculate its performance on the objective function;

[0027] F3. Selection operation: select excellent individuals as parents according to their fitness values;

[0028] F4. Crossover operation: Perform crossover operation on different parent individuals to generate new offspring individuals;

[0029] F5. Mutation operation: Perform mutation operation on offspring individuals to randomly change some parameter values ​​of offspring individuals to increase the diversity of the population;

[0030] F6. Evolutionary iteration: Add newly generated offspring individuals to the population, and eliminate offspring individuals that are below the performance standard based on fitness;

[0031] F7. Repeat steps F1 to F6 until a predetermined number of iterations or performance indicator is reached.

[0032] The beneficial effects of the present invention include:

[0033] 1. Improved the assessment accuracy of software reliability and the robustness of the model:

[0034] (1) Capturing complex nonlinear relationships: It can effectively capture complex nonlinear relationships in software reliability data. It can also adapt to multi-dimensional features and capture time-dependent changes by dynamically adjusting model parameters, significantly improving the prediction accuracy of the model.

[0035] (2) Reduce overfitting and enhance robustness: Regularization and cross-validation methods are used to prevent overfitting and improve the performance of the model on different data sets. The stability and robustness of the model are enhanced to ensure that consistent and reliable prediction results can be provided in practical applications.

[0036] 2. Improved computing efficiency and automation:

[0037] (1) Automated parameter optimization: Genetic algorithms are used for automated parameter optimization. The best parameter combination is quickly found through global search, which improves the efficiency of model training. The advantage of genetic algorithms lies in their powerful global search capabilities, which can effectively avoid local optimal solutions and ensure that the global optimal solution is found.

[0038] (2) Reduce manual intervention: The automated optimization process reduces the time and effort of manual parameter adjustment, making model building more efficient and convenient. The automated process ensures the consistency and accuracy of parameter adjustment, improving the efficiency and effectiveness of model development.

[0039] 3. Enhanced result interpretation and data utilization efficiency:

[0040] (1) Strong interpretability of results: The piecewise linear function and adaptive selection of segmentation points of the multivariate smooth spline regression model make the model output easy to understand and interpret. The transparent model structure and clear decision-making basis improve the trust and interpretability of the model results.

[0041] (2) Efficient use of multidimensional data and time series: By processing high-dimensional data features and adapting to the dynamic changes of time series data, the multidimensional information and time dependency in the data are fully utilized, thus improving data utilization. By fully mining and utilizing the data, it is ensured that the model can accurately reflect the software reliability characteristics and provide more valuable evaluation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 The present invention is a flowchart of a method for constructing a software reliability assessment model. DETAILED DESCRIPTION

[0043] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0044] like Figure 1 As shown, the method for constructing a software reliability evaluation model of the present invention comprises the steps of:

[0045] A. Obtain data from multiple sources, including:

[0046] Development process data: Obtain information such as code submission records, change history, developer activities, etc. from version control systems (such as Git).

[0047] Test data: Collect test cases, test results, defect reports and other data from test management tools (such as JIRA, TestRail).

[0048] Operation logs: Obtain operation logs, error logs, performance indicators, and other information from the log management system (such as ELK Stack) in the production environment.

[0049] User feedback data: collect user-reported bugs, user experience feedback, and other information from user feedback systems (such as Zendesk and UserVoice).

[0050] Then, the multi-source data is preprocessed, including:

[0051] Missing value processing: interpolate, fill or delete missing data to ensure data integrity.

[0052] Outlier detection: Detect and handle anomalous data points using statistical methods and machine learning algorithms.

[0053] Cross-source data alignment: Data alignment is performed based on timestamps and other key attributes (such as module name, function point, etc.) to integrate relevant information from different data sources.

[0054] Feature extraction: Extract key features such as code complexity indicators, test coverage, defect density, mean time to repair, user satisfaction scores, etc.

[0055] Feature correlation analysis: Analyze the relationship between each feature and software reliability through correlation coefficient, mutual information and other methods to screen out important features.

[0056] Feature dimensionality reduction processing: Use dimensionality reduction methods such as principal component analysis (PCA) and linear discriminant analysis (LDA) to reduce feature dimensions and improve model training efficiency.

[0057] Data normalization: Normalize numerical features to make them have the same scale.

[0058] Encoding processing: Encode categorical features (such as one-hot encoding and label encoding) to facilitate model processing.

[0059] B. Construct a multivariate smooth spline regression model. Assume that there are p features X1, X2, ..., X that affect software reliability. p , the target variable is Y (such as software failure rate). Use smooth spline functions to capture the nonlinear relationship between features and target variables:

[0060]

[0061] Among them, f i (Xi) is the smooth spline function, ∈ is the error term. Then the B-spline basis function is used as the spline basis function to obtain the final multivariate smooth spline regression model representation:

[0062]

[0063] Among them, B ij (X i ) is the jth B-spline basis function of the ith feature, β ij is the corresponding coefficient, k i is the number of basis functions of the i-th feature, β0 is the intercept term, which is a constant term in the multivariate smoothing spline regression model, used to balance the model so that it still has reasonable expected output when the eigenvalue is zero.

[0064] Spline basis functions, especially B-spline basis functions, are important tools in numerical analysis and computer graphics for creating smooth curves and surfaces. Basis functions have the characteristics of non-negativity and local support, which means that they are only effective within a specific interval, which makes them more flexible and efficient in design and editing. B-spline basis functions are determined by a set of control points and an order, and each basis function corresponds to a control point. Through the combination of these basis functions, the shape can be adjusted and refined while maintaining the smoothness and continuity of the curve.

[0065] C. Update the parameters of the multivariate smooth spline regression model in real time based on the prior distribution and the real-time data obtained during the program operation through the Bayesian formula. Assume that the prior distribution is P(θ) and the running data is D new , then the posterior distribution is:

[0066] P(θ|D new )∝P(D new |θ)P(θ);

[0067] Among them, θ is the model parameter, P(D new |θ) is the likelihood function.

[0068] Use the Bayesian formula to calculate the posterior distribution P(θ|D new ), update the parameters of the multivariate smoothing spline regression model.

[0069] D. Then, the parameters of the multivariate smooth spline regression model are updated in real time through Kalman filtering technology. Kalman filtering technology is a recursive algorithm that can perform parameter prediction, prediction error covariance calculation, Kalman gain calculation, state update (i.e. parameter update) and error covariance update of the multivariate smooth spline regression model in sequence every time running data arrives. Specifically:

[0070] a) Parameter prediction:

[0071]

[0072] Among them, A is the state transfer matrix, B is the control matrix, and u k is the control variable, k is the time step, indicating the current time step. Whenever new running data arrives, the current time step k increases by 1. is the estimated value of the state variable, It represents the estimated value of the model parameters at time step k after predicting them based on the model parameters at time step k-1. Time step k|k-1 means that the posterior estimate at time step k is calculated based on the prior estimate at time step k-1, and at time step k, the prior state at time step k-1 is used to update the state estimate at the current moment.

[0073] b) Forecast error covariance:

[0074] P k|k-1 =AP k-1|k-1 A T +Q;

[0075] Among them, P k-1|k-1 represents the estimated value of the error covariance matrix of the model state variables at time step k based on all available information before time step k-1, and Q is the process noise covariance matrix.

[0076] c) Kalman gain:

[0077] K k =P k|k-1 H T (HP k|k-1 H T +R) -1 ;

[0078] Where H is the observation matrix and R is the observation noise covariance matrix.

[0079] d) Status update:

[0080]

[0081] in, represents the updated estimate of the model state variable at time step k based on all available information before time step k, representing the posterior estimate at the kth time step. is the latest state update at the kth time step based on the data obtained from all previous time steps (i.e., from the 1st time step to the kth time step); k is the observed value.

[0082] e) Error covariance update:

[0083] P k|k =(IK k H)P k|k-1 ;

[0084] Where I is the identity matrix.

[0085] E. Train and verify the updated multivariate smooth spline regression model. The training and verification data sets are mainly obtained from the software testing phase, including software testing data, software operation data, log data, etc. All the acquired data are divided into 80% training set and 20% verification set. During training, the K-fold crossover method is used to train the model for different software development life cycle stages (such as development stage, testing stage, and operation stage), so that the model in each stage can accurately reflect the current reliability status.

[0086] When validating the trained multivariate smooth spline regression model, the K-fold cross-validation method is used to evaluate the model performance to ensure the generalization ability and robustness of the model. The multi-source data is randomly divided into k subsets of equal size, iterated k times, and a different subset is selected each time as the validation set, and the remaining k-1 subsets are used as training sets; in each iteration, the multivariate smooth spline regression model is trained using the training set, and the performance of the multivariate smooth spline regression model is evaluated on the validation set, and the average and standard deviation of the performance indicators (such as MSE (mean square error), accuracy, etc.) of the k-times validation are calculated as the final evaluation result of the model.

[0087] Through training and verification, the final evaluation model is obtained.

[0088] F: Finally, in view of the high-dimensional and non-convex optimization characteristics of software reliability evaluation, a genetic algorithm is used to automatically optimize the parameters of the evaluation model. The specific steps include:

[0089] F1. Initialize the population: Initialize a group of randomly generated individuals, each of which consists of a set of hyperparameters (in the context of machine learning, hyperparameters are parameters whose values ​​are set before starting the learning process, rather than parameter data obtained through training. Typically, hyperparameters need to be optimized to select a set of optimal hyperparameters for the learning machine to improve learning performance and results).

[0090] F2. Fitness evaluation: Evaluate each individual and calculate its performance on the objective function, such as the prediction accuracy of the model.

[0091] F3. Selection operation: Select excellent individuals as parents according to their fitness values. Commonly used methods include roulette selection, tournament selection, etc.

[0092] F4. Crossover operation: Perform crossover operation on different parent individuals to generate new offspring individuals. The crossover operation can be single-point crossover, multi-point crossover, etc.

[0093] F5. Mutation operation: Perform mutation operation on offspring individuals to randomly change some parameter values ​​(genes) of offspring individuals to increase the diversity of the population.

[0094] F6. Evolutionary iteration: Add newly generated offspring individuals to the population and eliminate offspring individuals that are below the performance standard based on their fitness.

[0095] F7. Repeat steps F1 to F6 until a predetermined number of iterations or performance indicator is reached.

[0096] Finally, the optimal software reliability evaluation model is obtained.

[0097] The above embodiments only express the specific implementation methods of the present application, and the description is relatively specific and detailed, but it cannot be understood as limiting the scope of the present application. For ordinary technicians in this field, several modifications and improvements made without departing from the concept of the present application all belong to the protection scope of the present application.

Claims

1. Method for constructing software reliability assessment model, Its characteristics are: including the steps of: A. Acquire multi-source data and pre-process the multi-source data; B. Construct a multivariate smoothing spline regression model; C. Update the parameters of the multivariate smooth spline regression model in real time based on the prior distribution and the operating data obtained in real time during the program operation through the Bayesian formula; D. then updating the parameters of the multivariate smooth spline regression model in real time through the Kalman filtering technology, and performing parameter prediction, prediction error covariance calculation, Kalman gain calculation, state update and error covariance update of the multivariate smooth spline regression model in sequence each time the operation data arrives; E. Train and validate the updated multivariate smooth spline regression model to obtain the final evaluation model.

2. The method for constructing a software reliability assessment model according to claim 1, characterized in that: The multi-source data in step A includes: development process data, test data, operation logs and user feedback data.

3. The method for constructing a software reliability assessment model according to claim 1, characterized in that: In step A, the preprocessing includes: missing value processing, outlier detection, cross-source data alignment, feature extraction, feature correlation analysis, feature dimensionality reduction processing, data normalization and encoding processing.

4. The method for constructing a software reliability assessment model according to claim 1, characterized in that: In step B, the nonlinear relationship between the features in the multi-source data and the target variable is captured by the smooth spline function to obtain the final multivariate smooth spline regression model.

5. The method for constructing a software reliability assessment model according to claim 1, characterized in that: In step C, the Bayesian formula is used to calculate the posterior distribution through the prior distribution and the likelihood function, and the parameters of the multivariate smooth spline regression model are updated in real time through the posterior distribution.

6. The method for constructing a software reliability assessment model according to claim 1, characterized in that: In step D, the state transfer matrix and control matrix in the Kalman filter technology are first determined, and then the state variables and error covariance are initialized. Then, each time operation data arrives, the parameter prediction, prediction error covariance calculation, Kalman gain calculation, state update and error covariance update of the multivariate smooth spline regression model are performed in sequence.

7. The method for constructing a software reliability assessment model according to claim 1, characterized in that: In step E, when training the updated multivariate smooth spline regression model, the K-fold cross validation method is used to train the model for different software development life cycle stages, so that the model of each stage can accurately reflect the current reliability status; when verifying the trained multivariate smooth spline regression model, the K-fold cross validation method is used to evaluate the model performance to ensure the generalization ability and robustness of the model and obtain the final evaluation model.

8. The method for constructing a software reliability assessment model according to claim 7, characterized in that: The multi-source data are randomly divided into k subsets of equal size, iterated k times, and a different subset is selected each time as a validation set, and the remaining k-1 subsets are used as training sets; in each iteration, the multivariate smooth spline regression model is trained using the training set, and the performance of the multivariate smooth spline regression model is evaluated on the validation set, and the average and standard deviation of the performance indicators of the k validations are calculated as the final evaluation result of the model.

9. The method for constructing a software reliability assessment model according to claim 1, characterized in that: Step F: Using a genetic algorithm to optimize the parameters of the evaluation model.

10. The method for constructing a software reliability assessment model as claimed in claim 9, characterized in that: F includes: F1. Initialize the population: Initialize a group of randomly generated individuals, each of which consists of a set of hyperparameters; F2. Fitness evaluation: Evaluate each individual and calculate its performance on the objective function; F3. Selection operation: select excellent individuals as parents according to their fitness values; F4. Crossover operation: Perform crossover operation on different parent individuals to generate new offspring individuals; F5. Mutation operation: Perform mutation operation on offspring individuals to randomly change some parameter values ​​of offspring individuals to increase the diversity of the population; F6. Evolutionary iteration: Add newly generated offspring individuals to the population, and eliminate offspring individuals that are below the performance standard based on fitness; F7. Repeat steps F1 to F6 until a predetermined number of iterations or performance indicator is reached.

Images