A software reliability evaluation method based on code metrics

Abstract

This invention discloses a software reliability assessment method based on code metrics, comprising the following steps: S1: Obtain project issue tickets and project commit records, mine the tracking relationship between issue tickets and commits, determine the trigger type of issue tickets, classify them into code-triggered and non-code-triggered types according to the triggering cause, and extract all project issue tickets with the trigger type being code-triggered. This invention is not limited by project runtime or the amount of fault data. It uses automated scanning tools to obtain code metrics as input during the code completion stage, predicting the software reliability level in advance, facilitating risk estimation and maintenance preparation for potential software failures at an early stage. Simultaneously, it considers the impact of code factors on reliability, analyzes the correlation between code metrics and reliability, provides developers with objective data to evaluate the quality of written code, and provides a reference for quality optimization.

Description

Technical Field

[0001] This invention relates to the field of software quality assessment technology, specifically to a software reliability assessment method based on code metrics. Background Technology

[0002] Software reliability refers to the probability that software will not fail within a specified time. The main goal of software reliability research is to build models to observe the normal operation of software over a past period and predict the probability of it not failing within a future period. Through reliability prediction, project teams can anticipate whether the software is prone to failure in the future, estimate maintenance workload, and mitigate the risk of software failure.

[0003] Currently, the mainstream software reliability assessment model is the incremental software reliability model. This model models reliability from the perspective of software failure, using mathematical methods, primarily differential equations (or systems of differential equations), to establish a quantitative function model between several random parameters in the software testing process. Using recorded software failure data, it fits the trend of the cumulative number of failures over time, and uses a specific formula to normalize the increasing number of failures into a reliability probability value, thus deriving a reliability calculation formula for a specific project. Given the model input—time nodes and time intervals—the reliability for a past or future time period can be calculated.

[0004] This method involves a function fitting process and can only be implemented after the project has been running for a period of time and a certain amount of accumulated fault data has been collected. It cannot be used for reliability prediction in the early stages of a project. At the same time, the incremental reliability model based on fault data does not involve the causes of faults and cannot fundamentally analyze and understand the model's prediction results.

[0005] To address the shortcomings of existing models, this paper proposes a software reliability assessment method based on code metrics. This method considers the impact of code factors on reliability and is applicable to cross-project data where data is missing and incremental software reliability model function fitting is not possible. It enables early prediction at the beginning of the project and during the code completion stage for code quality assessment and optimization. Summary of the Invention

[0006] The purpose of this invention is to provide a software reliability assessment method based on code metrics to solve the problems mentioned in the background art.

[0007] To address the aforementioned technical problems, this invention provides the following technical solution: a software reliability assessment method based on code metrics, characterized by comprising the following steps:

[0008] S1: Obtain project issue tickets and project commit records, explore the tracking relationship between project issue tickets and commits, determine the trigger type of project issue tickets, classify them into code-triggered and non-code-triggered types according to the triggering reason, and extract all project issue tickets with the trigger type of code-triggered.

[0009] S2: Based on the incremental software reliability model, regression analysis is used to obtain the trend and function of software reliability evolution with project version, thereby quantifying the corresponding reliability values ​​for each version of the project.

[0010] S3: Obtain the source code of each version of the project and use a code metric scanning tool to calculate the code metrics corresponding to each version of the project.

[0011] S4: Perform reliability analysis on the code metrics corresponding to each version of the project and select the model feature set;

[0012] S5: Divide reliability values ​​into four categories as target features, divide the model feature set into training set and test set, and train the four-classification model through the training set and test set to build a software reliability assessment model based on the code indicators corresponding to each version of the project.

[0013] S6: Input the code metrics corresponding to the version of the project to be evaluated into the software reliability assessment model. The software reliability assessment model outputs the reliability label of the code of the version of the project to be evaluated, predicts reliability during the code completion stage, and assists in code quality assessment and optimization.

[0014] Furthermore, S1 includes the following steps:

[0015] S1-1: Select project failure issue tickets and commit data, explore the tracking relationship between the two, and divide the triggering factors of project failure issue tickets into code-triggered and non-code-triggered types in order to extract the code-triggered issue tickets required for the experiment;

[0016] If the commit data contains a field indicating the issue number to which it belongs, then that field is used to associate it with the issue number field. The issue involved or resolved in the commit is a code-triggered issue.

[0017] If the commit data does not have a field to indicate the issue number, but mentions it in the fault description, then the commit description text information is used. The issue number mentioned in the fault description is searched based on rule matching, and the issue number is associated with the issue number field to indirectly find code-triggered issue issues and mark them as non-code-triggered issue issues.

[0018] S1-2: Label all issue tickets using direct and indirect methods, with labels indicating whether they are code-triggered or not. Extract code-triggered issue tickets as data input for the software reliability incremental model.

[0019] Furthermore, S2 includes the following steps:

[0020] S2-1: Using the code-triggered project fault issue slips extracted in step S1, use the software reliability incremental model to fit the project reliability value as a function of version change.

[0021] S2-2: Based on the obtained reliability value of the project as a function of version change, calculate the reliability value of each version of the project and summarize the statistics.

[0022] Furthermore, S3 includes the following steps:

[0023] S3-1: Obtain the source code for each version of the project;

[0024] S3-2: Use plugins / code scanning tools to scan the code metrics corresponding to each version of the project and summarize the statistics.

[0025] This invention can utilize plugins / code scanning tools including Metrics, Source Meter, and Metric Tree to calculate source code metrics from multiple dimensions such as packages, classes, and files. Note that this invention requires obtaining overall version code metrics; therefore, careful attention must be paid to the calculation method when summarizing. Cumulative metrics (such as lines of code, number of methods, etc.) need to be calculated cumulatively, while ratio metrics (such as clone coverage, etc.) can only be calculated as an average.

[0026] Furthermore, S4 includes the following steps:

[0027] S4-1: Use steps S2 and S3 to summarize the reliability values ​​of each version of the project and the corresponding code metrics of each version of the project, and analyze the code metrics that affect the reliability of the project.

[0028] In analyzing the relationship between code metrics and reliability, three feature selection schemes were used: Pearson correlation coefficient, chi-square test, and information gain.

[0029] The Pearson correlation coefficient, as described in this invention, is one of the simplest methods to help understand the relationship between features and response variables. This method measures the linear correlation between variables, with values ​​ranging from -1 to 1. Positive values ​​indicate a positive correlation, negative values ​​indicate a negative correlation, and larger absolute values ​​indicate a stronger linear correlation. The Pearson correlation coefficient between two variables is defined as the quotient of their covariance and standard deviation.

[0030]

[0031] Pearson correlation coefficient is widely used, easy to calculate, and helps in the initial understanding of data;

[0032] The chi-square test (CHI) infers whether there is a significant difference between the population distribution and the expected distribution based on sample data. It is applied to the analysis of count data, makes no assumptions about the population distribution, and is a type of nonparametric test. Assuming the independent variable has N possible values ​​and the dependent variable has M possible values, it considers the difference between the observed and expected frequencies of samples where the independent variable equals i and the dependent variable equals j, and constructs the statistic:

[0033]

[0034] Where A is the actual value and E is the theoretical value.

[0035] In Information Gain (IG), importance is measured by how much information a feature brings to the classification system. Information gain is used to measure this information; a higher information gain indicates a more important feature. The specific formula is as follows:

[0036] IG(T) = H(C) - H(C|T)

[0037] This formula represents the difference between the original entropy of the system and the conditional entropy after fixing feature T. Here, H(C) is the entropy of the system, and H(C|T) is the entropy of the system under condition T. The specific calculation formula is as follows:

[0038]

[0039]

[0040] S4-2: Analyze the reliability of the project using three different approaches, and use the set of code metrics that have the greatest impact on reliability as the model feature set;

[0041] In order to extract a set of code metrics suitable for training a reliability model and to avoid the limitations of a certain feature selection method, feature selection is performed using all three methods. The results of the feature selection are then analyzed to obtain the final set of code metrics.

[0042] Furthermore, S5 includes the following steps:

[0043] S5-1: Classify the software reliability values ​​as follows: [0,0.25] indicates low reliability, (0.25,0.5] indicates relatively low reliability, (0.5,0.75] indicates relatively high reliability, and (0.75,1] indicates high reliability.

[0044] S5-2: Data preprocessing, which is to preprocess the features obtained from the model feature set. The preprocessing includes outlier handling, missing value handling, data normalization, and continuous feature processing.

[0045] The features obtained in this invention are obtained by comprehensively analyzing the reliability values ​​and code indicators of each version of the project using three feature selection methods, ensuring that the features included in the final feature set are ranked highly in the calculation results of the three methods.

[0046] S5-3: A random forest classifier is selected to train the four-class classification model, and K-fold cross-validation is used to validate the classifier.

[0047] In this invention, random forest is a supervised learning method that is an extension of decision tree. It reduces the overfitting problem in a single decision tree by combining multiple decision trees. It is simple to implement, has strong generalization ability, and can handle high-dimensional data. It is usually suitable for classification and regression tasks.

[0048] K-fold cross-validation is a method for estimating the performance of machine learning models, suitable for situations with limited data. The original data D is randomly divided into K parts. Each time, (K-1) parts are selected as the training set, and the remaining part is used as the test set. Cross-validation is repeated K times, and the average accuracy of the K iterations is taken as the final model's evaluation metric.

[0049] Furthermore, S6 includes the following steps:

[0050] S6-1: Select the version of the project to be evaluated, use a code scanning tool to calculate the code metrics corresponding to the version of the project to be evaluated, input them into the software reliability assessment model, and obtain the predicted reliability.

[0051] S6-2: Analyze code metrics based on the predicted reliability level and assist in code quality assessment and optimization.

[0052] Furthermore, step S6 applies to both data integrity projects and data missing projects.

[0053] When the project to be evaluated is a data complete project, use steps S1 to S5 to train a classification model and use these steps to evaluate the reliability of the version code of the project to be evaluated.

[0054] When the project to be evaluated is a data-missing project, a model obtained from a data-complete project of similar size can be used, omitting steps S1 to S5, and directly using this step for reliability assessment.

[0055] Furthermore, the incremental software reliability model models reliability from the perspective of software failure. It uses mathematical methods, primarily differential equations, to establish quantitative function models between several random parameters in the software testing process. Among these quantitative function models, the SRGM of the non-homogeneous Poisson process (NHPP) is the most widely used, and it satisfies the following basic common assumptions:

[0056] (1) Assuming that failure events occur randomly, the observation of failures and troubleshooting by testers and troubleshooters meet the requirements of NHPP;

[0057] (2) Let [N(t), t≥0] be a random counting process, N(t) be the cumulative number of faults detected by the tester within [0,t], and E[N(t)]=m(t), where m(t) is the mean function, satisfying m(0)=0.

[0058] The following formula is obtained:

[0059]

[0060]

[0061] λ(t) = b(t)[a(t) - m(t)]

[0062] P{N(t)=n} is the function that changes the reliability value of the project with the version, E[N(t)] and m(t) are the expected cumulative number of failures at time t, λ(t) is the failure occurrence rate at time t, a(t) is the failure function related to t, and b(t) is the failure detection rate function related to t.

[0063] The software reliability R(x|t) can be calculated using the following formula. If we let t (t≥0, x>0) be the time point when the last failure occurred, then the software reliability in the interval (t, t+x) is:

[0064] R(x|t)=e -(m(t+x)-m(t))

[0065] Since the release intervals for each version of the project are the same, they can be regarded as uniformly changing variables. By replacing the independent variable t in the formula with the version number and the variable x in the formula with the release interval of the versions, curve fitting is performed to obtain the function of project reliability changing with version.

[0066] Compared with existing technologies, the beneficial effects achieved by this invention are as follows: This invention is not limited by project runtime or the amount of fault data. It uses automated scanning tools to obtain code metrics as input during the code completion phase, predicting software reliability in advance and facilitating early risk assessment and maintenance preparation for potential software failures. Simultaneously, it considers the impact of code factors on reliability, analyzes the correlation between code metrics and reliability, provides developers with objective data to evaluate code quality, and offers a reference for quality optimization. Attached Figure Description

[0067] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0068] Figure 1 This is a flowchart illustrating a software reliability assessment method based on code metrics according to the present invention.

[0069] Figure 2 This is a schematic diagram illustrating the process of applying the incremental software reliability model in this embodiment;

[0070] Figure 3 This is a schematic diagram of the model training process;

[0071] Figure 4 This is a diagram illustrating the process of optimizing code quality in a real-world application model. Detailed Implementation

[0072] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0073] Please see Figure 1-4 This invention provides a technical solution: a software reliability assessment method based on code metrics, characterized by comprising the following steps:

[0074] S1: Obtain project issue tickets and project commit records, explore the tracking relationship between project issue tickets and commits, determine the trigger type of project issue tickets, classify them into code-triggered and non-code-triggered types according to the triggering reason, and extract all project issue tickets with the trigger type of code-triggered.

[0075] S2: Based on the incremental software reliability model, regression analysis is used to obtain the trend and function of software reliability evolution with project version, thereby quantifying the corresponding reliability values ​​for each version of the project.

[0076] S3: Obtain the source code of each version of the project and use a code metric scanning tool to calculate the code metrics corresponding to each version of the project.

[0077] S4: Perform reliability analysis on the code metrics corresponding to each version of the project and select the model feature set;

[0078] S5: Divide reliability values ​​into four categories as target features, divide the model feature set into training set and test set, and train the four-classification model through the training set and test set to build a software reliability assessment model based on the code indicators corresponding to each version of the project.

[0079] S6: Input the code metrics corresponding to the version of the project to be evaluated into the software reliability assessment model. The software reliability assessment model outputs the reliability label of the code of the version of the project to be evaluated, predicts reliability during the code completion stage, and assists in code quality assessment and optimization.

[0080] Furthermore, S1 includes the following steps:

[0081] S1-1: Select project failure issue tickets and commit data, explore the tracking relationship between the two, and divide the triggering factors of project failure issue tickets into code-triggered and non-code-triggered types in order to extract the code-triggered issue tickets required for the experiment;

[0082] If the commit data contains a field indicating the issue number to which it belongs, then that field is used to associate it with the issue number field. The issue involved or resolved in the commit is a code-triggered issue.

[0083] If the commit data does not have a field to indicate the issue number, but mentions it in the fault description, then the commit description text information is used. The issue number mentioned in the fault description is searched based on rule matching, and the issue number is associated with the issue number field to indirectly find code-triggered issue issues and mark them as non-code-triggered issue issues.

[0084] S1-2: Label all issue tickets using direct and indirect methods, with labels indicating whether they are code-triggered or not. Extract code-triggered issue tickets as data input for the software reliability incremental model.

[0085] Furthermore, S2 includes the following steps:

[0086] S2-1: Using the code-triggered project fault issue slips extracted in step S1, use the software reliability incremental model to fit the project reliability value as a function of version change.

[0087] S2-2: Based on the obtained reliability value of the project as a function of version change, calculate the reliability value of each version of the project and summarize the statistics.

[0088] Furthermore, S3 includes the following steps:

[0089] S3-1: Obtain the source code for each version of the project;

[0090] S3-2: Use plugins / code scanning tools to scan the code metrics corresponding to each version of the project and summarize the statistics.

[0091] This invention can utilize plugins / code scanning tools including Metrics, Source Meter, and Metric Tree to calculate source code metrics from multiple dimensions such as packages, classes, and files. Note that this invention requires obtaining overall version code metrics; therefore, careful attention must be paid to the calculation method when summarizing. Cumulative metrics (such as lines of code, number of methods, etc.) need to be calculated cumulatively, while ratio metrics (such as clone coverage, etc.) can only be calculated as an average.

[0092] Furthermore, S4 includes the following steps:

[0093] S4-1: Use steps S2 and S3 to summarize the reliability values ​​of each version of the project and the corresponding code metrics of each version of the project, and analyze the code metrics that affect the reliability of the project.

[0094] In analyzing the relationship between code metrics and reliability, three feature selection schemes were used: Pearson correlation coefficient, chi-square test, and information gain.

[0095] The Pearson correlation coefficient, as described in this invention, is one of the simplest methods to help understand the relationship between features and response variables. This method measures the linear correlation between variables, with values ​​ranging from -1 to 1. Positive values ​​indicate a positive correlation, negative values ​​indicate a negative correlation, and larger absolute values ​​indicate a stronger linear correlation. The Pearson correlation coefficient between two variables is defined as the quotient of their covariance and standard deviation.

[0096]

[0097] Pearson correlation coefficient is widely used, easy to calculate, and helps in the initial understanding of data;

[0098] The chi-square test (CHI) infers whether there is a significant difference between the population distribution and the expected distribution based on sample data. It is applied to the analysis of count data, makes no assumptions about the population distribution, and is a type of nonparametric test. Assuming the independent variable has N possible values ​​and the dependent variable has M possible values, it considers the difference between the observed and expected frequencies of samples where the independent variable equals i and the dependent variable equals j, and constructs the statistic:

[0099]

[0100] Where A is the actual value and E is the theoretical value.

[0101] In Information Gain (IG), importance is measured by how much information a feature brings to the classification system. Information gain is used to measure this information; a higher information gain indicates a more important feature. The specific formula is as follows:

[0102] IG(T) = H(C) - H(C|T)

[0103] This formula represents the difference between the original entropy of the system and the conditional entropy after fixing feature T. Here, H(C) is the entropy of the system, and H(C|T) is the entropy of the system under condition T. The specific calculation formula is as follows:

[0104]

[0105]

[0106] S4-2: Analyze the reliability of the project using three different approaches, and use the set of code metrics that have the greatest impact on reliability as the model feature set;

[0107] In order to extract a set of code metrics suitable for training a reliability model and to avoid the limitations of a certain feature selection method, feature selection is performed using all three methods. The results of the feature selection are then analyzed to obtain the final set of code metrics.

[0108] Furthermore, S5 includes the following steps:

[0109] S5-1: Classify the software reliability values ​​as follows: [0,0.25] indicates low reliability, (0.25,0.5] indicates relatively low reliability, (0.5,0.75] indicates relatively high reliability, and (0.75,1] indicates high reliability.

[0110] S5-2: Data preprocessing, which is to preprocess the features obtained from the model feature set. The preprocessing includes outlier handling, missing value handling, data normalization, and continuous feature processing.

[0111] S5-3: A random forest classifier is selected to train the four-class classification model, and K-fold cross-validation is used to validate the classifier.

[0112] In this invention, random forest is a supervised learning method that is an extension of decision tree. It reduces the overfitting problem in a single decision tree by combining multiple decision trees. It is simple to implement, has strong generalization ability, and can handle high-dimensional data. It is usually suitable for classification and regression tasks.

[0113] K-fold cross-validation is a method for estimating the performance of machine learning models, suitable for situations with limited data. The original data D is randomly divided into K parts. Each time, (K-1) parts are selected as the training set, and the remaining part is used as the test set. Cross-validation is repeated K times, and the average accuracy of the K iterations is taken as the final model's evaluation metric.

[0114] Furthermore, S6 includes the following steps:

[0115] S6-1: Select the version of the project to be evaluated, use a code scanning tool to calculate the code metrics corresponding to the version of the project to be evaluated, input them into the software reliability assessment model, and obtain the predicted reliability.

[0116] S6-2: Analyze code metrics based on the predicted reliability level and assist in code quality assessment and optimization.

[0117] Furthermore, step S6 applies to both data integrity projects and data missing projects.

[0118] When the project to be evaluated is a data complete project, use steps S1 to S5 to train a classification model and use these steps to evaluate the reliability of the version code of the project to be evaluated.

[0119] When the project to be evaluated is a data-missing project, a model obtained from a data-complete project of similar size can be used, omitting steps S1 to S5, and directly using this step for reliability assessment.

[0120] Furthermore, the incremental software reliability model models reliability from the perspective of software failure. It uses mathematical methods, primarily differential equations, to establish quantitative function models between several random parameters in the software testing process. Among these quantitative function models, the SRGM of the non-homogeneous Poisson process (NHPP) is the most widely used, and it satisfies the following basic common assumptions:

[0121] (1) Assuming that failure events occur randomly, the observation of failures and troubleshooting by testers and troubleshooters meet the requirements of NHPP;

[0122] (2) Let [N(t), t≥0] be a random counting process, N(t) be the cumulative number of faults detected by the tester within [0,t], and E[N(t)]=m(t), where m(t) is the mean function, satisfying m(0)=0.

[0123] The following formula is obtained:

[0124]

[0125]

[0126] λ(t) = b(t)[a(t) - m(t)]

[0127] P{N(t)=n} is the function that changes the reliability value of the project with the version, E[N(t)] and m(t) are the expected cumulative number of failures at time t, λ(t) is the failure occurrence rate at time t, a(t) is the failure function related to t, and b(t) is the failure detection rate function related to t.

[0128] The software reliability R(x|t) can be calculated using the following formula. If we let t (t≥0, x>0) be the time point when the last failure occurred, then the software reliability in the interval (t, t+x) is:

[0129] R(x|t)=e -(m(t+x)-m(t))

[0130] Since the release intervals for each version of the project are the same, they can be regarded as uniformly changing variables. By replacing the independent variable t in the formula with the version number and the variable x in the formula with the release interval of the versions, curve fitting is performed to obtain the function of project reliability changing with version.

[0131] Example 1

[0132] This embodiment primarily describes software reliability assessment based on code metrics for projects with a certain amount of data accumulation and runtime, i.e., when fault data and commit data are sufficient and complete. Sufficient data here means the project has released several versions, each with corresponding recorded fault and commit data, supporting subsequent fitting steps. Complete data means the fault data and committed information have complete, defined fields to label the issue numbers involved, and the submitter labels them as required during implementation. In this case, the labeling of code-triggered issues can be directly completed by associating these issue fields with the issue data. This embodiment mines project fault issues, commit data, and code repository source code, uses an incremental software reliability model to calculate reliability values ​​as intermediate auxiliary results, associates them with scanned code metrics, constructs a model, and predicts reliability during the code submission phase.

[0133] Step 1: Obtain project issue tickets, project commit data, and extract code that triggers issue tickets.

[0134] Issue tickets are software errors discovered and reported during project execution. They typically record the time of the error, a description of the error, and the personnel responsible for resolving it. Project commit data contains information recorded by developers when they commit or modify code. It usually includes the content of the commit or modification, the issue tickets involved, and the personnel responsible for the modification.

[0135] In this case study, the enterprise project commit data has a clearly marked field indicating the issue ticket number involved in each commit. The solution is to associate this field with the issue ticket number field in the commit data. Specifically, for each issue ticket, the commit data is scanned for the issue ticket number field. If a commit with that issue ticket number exists, the issue ticket is marked as "code-triggered"; otherwise, it is marked as "non-code-triggered." All issue tickets marked as "code-triggered" are then filtered out and used in subsequent steps.

[0136] Step 2: Quantify the reliability values ​​of each version based on the incremental software reliability model.

[0137] This invention aims to predict reliability during the code completion phase, unaffected by project runtime or the number of failures. The purpose of using an incremental software reliability model is to quantify reliability values ​​and explore the correlation between reliability and code metrics in order to build a predictive model. This step serves only as an intermediate result to assist model training; once the model is built, it can be used for cross-project data where problem tickets are missing.

[0138] The main principle of the incremental software reliability model is to fit a trend function of the cumulative number of failures over time and map it to a reliability value between 0 and 1 using the following formula. Over the decades, researchers have proposed many function models considering various factors in the failure detection and repair process. Table 1 only lists a few frequently discussed and applied models.

[0139] R(x|t)=e -(m(t+x)-m(t))

[0140] Table 1

[0141]

[0142]

[0143] Where a(t) is the fault function related to t, and b(t) is the fault detection rate function related to t. The key to data fitting is to obtain these two parameters.

[0144] Basic Model 1 is an exponential growth model that treats the fault detection rate as a fixed value. This model sometimes fails to accurately analyze actual project error data for software quality assessment.

[0145] Models 2, 3, and 4 are widely used S-Shaped models that assume the fault detection rate changes over time, describing the software error elimination phenomenon and its physical explanation. These models possess a time delay function in the sense of control theory. At the beginning of the testing phase, some faults may be covered by other faults; these covered faults remain undetected until they are actually eliminated.

[0146] Model 3 describes a software fault detection phenomenon that is interdependent with the detected errors. This model has been widely used by Japanese computer manufacturers to assess the reliability growth of their software products. In the implementation of this enterprise case, Model 3 was observed to have the best fit.

[0147] The specific implementation details of this step are as follows:

[0148] 1) Basic assumptions: The version intervals are roughly uniform, and versions are updated at roughly the same time.

[0149] 2) Use the project version number and the number of faults in each version to fit the model. The resulting model can describe the trend of reliability change as the version number increases.

[0150] 3) Quantify the reliability of each version.

[0151] The incremental software reliability model uses time *t* and time interval *x* as its functional variables. This invention requires linking reliability with code metrics; therefore, a new scheme is proposed, replacing time *t* with version number *v* and time interval *x* with version interval *c*. Based on company regulations and version release patterns, the intervals between releases are roughly the same, conforming to the uniform characteristics of time progression. Furthermore, the scheme shows good fitting results in practice, thus making it reasonable.

[0152] It is worth noting that while the incremental software reliability model can predict the reliability of future versions based on version number progression, it only considers actual fault exposure and remediation. This invention focuses on fundamentally analyzing the factors influencing reliability to assess code quality and assist developers in evaluating and improving their code performance.

[0153] Step 3: Obtain the project source code and calculate code metrics.

[0154] Download the source code of each version from the selected enterprise project. The version number must be consistent with the version number of the reliability calculation in step 2.

[0155] Use code scanning tools to scan for various code metrics and then summarize and statistically analyze them.

[0156] Step 4: Analyze the code metrics that affect the reliability level and select the feature set.

[0157] Steps 2 and 3 respectively yield the reliability values ​​and code metrics for each version of the selected project. In this embodiment, the initial code metric set is a summary of all metrics obtained by the code metric scanning tool. Three feature selection methods are used to comprehensively analyze and obtain the final result, ensuring that the features included in the final feature set are all ranked in the top 30% of the results calculated by the three methods. Table 2 lists the final selected features and their corresponding meanings.

[0158] Table 2

[0159]

[0160]

[0161] Step 5: Use discrete reliability values ​​as target features to train the model.

[0162] To construct the four-class classification model, the reliability values ​​need to be discretized: [0, 0.25] represents "low reliability", (0.25, 0.5] represents "relatively low reliability", (0.5, 0.75] represents "relatively high reliability", and (0.75, 1] ​​represents "high reliability". Table 3 shows the reliability values ​​and discretized labels for some versions of the selected items.

[0163] Table 3

[0164]

[0165] The acquired samples were trained using a random forest model, an ensemble machine learning algorithm derived from decision trees. Random forests improve classification performance by combining multiple decision trees to reduce overfitting issues common in single decision trees. K-fold cross-validation was then used for model tuning. The original data D was randomly divided into K parts, with (K-1) parts selected as the training set each time, and the remaining part used as the test set. Cross-validation was repeated K times, and the average accuracy of the K iterations was used as the final model evaluation metric. In this implementation, K = 10.

[0166] Classifier performance results are evaluated using accuracy, recall, and F1 score.

[0167] The accuracy rate is 78.54% (the ratio of the number of samples correctly classified by the classifier to the total number of samples for a given test dataset).

[0168] The recall rate was 70.26% (the percentage of positive cases in the sample that were correctly predicted).

[0169] The F1 score was 67.20% (a weighted average of precision and recall).

[0170] Step 6: Select the version of the project to be evaluated and input the model to assess reliability.

[0171] Select the project version to be evaluated, use a code scanning tool to calculate source code metrics, summarize them and input them into the model to output reliability labels. Table 4 shows the code metrics of the selected project version.

[0172] Table 4

[0173] Indicator Name LOC WMC CBO NOM RFC DIT NOC LCOM Indicator value 52689 4415 2734 2712 3413 150 116 515

[0174] The reliability metric for this version is "high reliability". A comparison of the metrics reveals that this version has lower cyclomatic complexity and lower coupling compared to other releases, meaning that the probability of errors and related errors is lower, resulting in higher reliability.

[0175] Example 2

[0176] This embodiment addresses projects in their initial stages, characterized by a limited number of version iterations and missing version code data and fault record data. This makes the fault data fitting step of traditional incremental software reliability models impossible to implement. This invention solves the reliability assessment problem for projects with missing data, which traditional models cannot cover. The model is trained using projects of similar size and with complete data (specific training steps are described in Embodiment 1). The resulting model is then applied across projects with missing data to estimate reliability in the early stages of code completion and predict the likelihood of later failures.

[0177] This embodiment directly executes step 6, which involves selecting the version of the project to be evaluated and inputting it into the model to assess reliability.

[0178] The project is in its initial iteration phase. When the code is completed, a static code scanning tool is used to scan for code metrics that are consistent with the model input and summarize them. Table 5 shows the values ​​of each code metric in the project.

[0179] Table 5

[0180] Indicator Name LOC WMC CBO NOM RFC DIT NOC LCOM Indicator value 63140 5385 3640 3573 4228 249 161 673

[0181] The scan results were input into the model, and the result was "low reliability".

[0182] Comparing metrics allows for root cause analysis of the prediction results. Compared to projects of similar size, this project's code version has a higher total complexity and coupling within its classes, making detection difficult and increasing the probability of associated failures, thus leading to lower reliability. The analysis results will also benefit subsequent code quality optimization.

[0183] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0184] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A software reliability assessment method based on code metrics, characterized in that, Includes the following steps: S1: Obtain project issue tickets and project commit records, explore the tracking relationship between project issue tickets and commits, determine the trigger type of project issue tickets, classify them into code-triggered and non-code-triggered types according to the triggering reason, and extract all project issue tickets with the trigger type of code-triggered. S2: Based on the incremental software reliability model, regression analysis is used to obtain the trend and function of software reliability evolution with project version, thereby quantifying the corresponding reliability values ​​for each version of the project. S3: Obtain the source code of each version of the project and use a code metric scanning tool to calculate the code metrics corresponding to each version of the project. S4: Perform reliability analysis on the code metrics corresponding to each version of the project and select the model feature set; S5: Divide reliability values ​​into four categories as target features, divide the model feature set into training set and test set, and train the four-classification model through the training set and test set to build a software reliability assessment model based on the code indicators corresponding to each version of the project. S6: Input the code metrics corresponding to the version of the project to be evaluated into the software reliability assessment model. The software reliability assessment model outputs the reliability label of the code of the version of the project to be evaluated, predicts reliability during the code completion stage, and assists in code quality assessment and optimization.

2. The software reliability assessment method based on code metrics according to claim 1, characterized in that, S1 includes the following steps: S1-1: Select project failure issue tickets and commit data, explore the tracking relationship between the two, and divide the triggering factors of project failure issue tickets into code-triggered and non-code-triggered types in order to extract the code-triggered issue tickets required for the experiment; If the commit data contains a field indicating the issue number to which it belongs, then that field is used to associate it with the issue number field. The issue involved or resolved in the commit is a code-triggered issue. If the commit data does not have a field to indicate the issue number, but mentions it in the fault description, then the commit description text information is used. The issue number mentioned in the fault description is searched based on rule matching, and the issue number is associated with the issue number field to indirectly find code-triggered issue issues and mark them as non-code-triggered issue issues. S1-2: Label all issue tickets using direct and indirect methods, with labels indicating whether they are code-triggered or not. Extract code-triggered issue tickets as data input for the software reliability incremental model.

3. The software reliability assessment method based on code metrics according to claim 1, characterized in that, S2 includes the following steps: S2-1: Using the code-triggered project fault issue slips extracted in step S1, use the software reliability incremental model to fit the project reliability value as a function of version change. S2-2: Based on the obtained reliability value of the project as a function of version change, calculate the reliability value of each version of the project and summarize the statistics.

4. The software reliability assessment method based on code metrics according to claim 1, characterized in that, S3 includes the following steps: S3-1: Obtain the source code for each version of the project; S3-2: Use plugins / code scanning tools to scan the code metrics corresponding to each version of the project and summarize the statistics.

5. The software reliability assessment method based on code metrics according to claim 1, characterized in that, S4 includes the following steps: S4-1: Use steps S2 and S3 to summarize the reliability values ​​of each version of the project and the corresponding code metrics of each version of the project, and analyze the code metrics that affect the reliability of the project. In analyzing the relationship between code metrics and reliability, three feature selection schemes were used: Pearson correlation coefficient, chi-square test, and information gain. S4-2: Analyze the reliability of the project using three different approaches, and use the set of code metrics that have the greatest impact on reliability as the model feature set.

6. The software reliability assessment method based on code metrics according to claim 1, characterized in that, S5 includes the following steps: S5-1: Classify software reliability values ​​as follows: [0, 0.25] indicates low reliability, (0.25, 0.5] indicates relatively low reliability, (0.5, 0.75] indicates relatively high reliability, and (0.75, 1] ​​indicates high reliability. S5-2: Data preprocessing, which is to preprocess the features obtained from the model feature set. The preprocessing includes outlier handling, missing value handling, data normalization, and continuous feature processing. S5-3: A random forest classifier is selected to train the four-class classification model, and K-fold cross-validation is used to validate the classifier.

7. The software reliability assessment method based on code metrics according to claim 1, characterized in that, S6 includes the following steps: S6-1: Select the version of the project to be evaluated, use a code scanning tool to calculate the code metrics corresponding to the version of the project to be evaluated, input them into the software reliability assessment model, and obtain the predicted reliability. S6-2: Analyze code metrics based on the predicted reliability level and assist in code quality assessment and optimization.

8. The software reliability assessment method based on code metrics according to claim 1, characterized in that: Step S6 applies to both data complete projects and data missing projects. When the project to be evaluated is a data complete project, use steps S1 to S5 to train a four-class classification model, and use these steps to evaluate the reliability of the version code of the project to be evaluated. When the project to be evaluated is a data-missing project, a model obtained from a data-complete project of similar size can be used, omitting steps S1 to S5, and directly using this step for reliability assessment.

9. The software reliability assessment method based on code metrics according to claim 3, characterized in that: The incremental software reliability model models reliability from the perspective of software failure. It uses mathematical methods, primarily differential equations, to establish quantitative function models between several random parameters in the software testing process. Among these quantitative function models, the SRGM model of the non-homogeneous Poisson process (NHPP) is the most widely used, and it satisfies the following basic common assumptions: (1) Assuming that failure events occur randomly, the observation of failures and troubleshooting by testers and troubleshooters meet the requirements of NHPP; (2) Let [N(t), t≥0] be a random counting process, N(t) be the cumulative number of faults detected by the tester within [0, t], and E[N(t)]=m(t), where m(t) is the mean function, satisfying m(0)=0. The following formula is obtained: Let E[N(t)] be the expected cumulative number of failures at time t, λ(t) be the failure occurrence rate at time t, a(t) be the failure function related to time t, and b(t) be the failure detection rate function related to time t. The software reliability R(x|t) can be calculated using the following formula. If t is the time point of the last failure, t≥0, x>0, then the software reliability in the range (t, t+x) is: Since the release intervals for each version of the project are the same, they can be regarded as uniformly changing variables. By replacing the independent variable t in the formula with the version number and the variable x in the formula with the release interval of the versions, curve fitting is performed to obtain the function of project reliability changing with version.

Images