Open source big data system software reliability modeling method based on Weibull-Weibull distribution

Through the Weibull-Weibull distribution model, the problem of the increase in the number of failures in the later stage of development and testing of open source big data system software is solved, and better fault prediction performance and stability are achieved, which is better than the existing technology.

CN118964172APending Publication Date: 2024-11-15SHANXI UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410937002.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-12
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

The existing software reliability model is difficult to effectively model the significant increase in the number of failures in the late development and testing of open source big data system software in the cloud computing environment, and it is difficult to accurately predict the number of failures.

Method used

Adopt the software reliability model of the open source big data system based on Weibull-Weibull distribution. By assuming that the fault detection process obeys the non-homogeneous Poisson process, and using the least squares method to estimate the model parameters, the mathematical expression of the fault detection rate is derived to achieve the fault. Accurate prediction of quantity.

Benefits of technology

The model performed well in fitting and predictive performance experiments on six fault data sets and eight software reliability models, proving that it is better than the accuracy and stability of the remaining fault number during the software testing of open source big data systems Other models.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118964172A_ABST
    Figure CN118964172A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of computer software, and particularly relates to an open source big data system software reliability modeling method based on Weibull-Weibull distribution. Considering that the number of faults detected in the later test period of the open-source big data system software is obviously increased in the software development and test process, the invention provides the following assumptions: (1) the fault detection process of the open-source big data system software obeys the non-homogeneous Poisson process; (2) the found fault is eliminated immediately, and a new fault is not introduced; (3) the faults detected in the open source big data system software are related to the remaining faults in the software; and (4) the fault detection rate of the open source big data system software obeys Weibull-Weibull distribution. And establishing a software reliability model oriented to the actual development and test environment of the open source big data system software by adopting a direct modeling method.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of computer software technology, and in particular relates to an open source big data system software reliability modeling method based on Weibull-Weibull distribution. Background Art

[0002] In recent years, the rapid development of big data technology has brought great convenience to people's production and life. For example, open source big data system software plays an important role in the processing of massive data, especially in the fields of e-commerce, healthcare, public services, finance, etc.

[0003] Common open source big data system software (or components) include Hadoop, HDFS, Hive, Pig, Mahaut, MapReduce, YARN, Zookeeper, HBase, etc. These components work together to complete complex big data distributed processing tasks. The reliability of these components directly affects the performance and quality of big data processing. Therefore, the reliability of these components has also attracted people's attention.

[0004] Some scholars have proposed using software reliability models (SRMs) to evaluate the reliability of open source big data system software. For example, Tamura et al. studied the behavior of big data system failures in cloud computing environments, proposed a software reliability model based on jump diffusion, and used open source big data system software failure data sets to conduct model parameter estimation, fitting, and performance evaluation experiments. Considering the characteristics of big data and cloud computing, Tamura and Yamada proposed a software reliability model based on three-dimensional Wiener process and conducted corresponding experiments on big data failure data sets. Tamura et al. proposed a software reliability analysis method based on three-dimensional fault data clustering in big data and cloud computing environments. Considering the correlation between actual big data processing and cloud computing, Tamura et al. proposed a software reliability model based on three-dimensional stochastic differential equations by studying the changes in communication traffic between big data and cloud computing. Tamura and Yamada proposed a software reliability model with irregular changes in fault detection rate through their research on cloud computing and big data. Considering that big data processing requires a cloud computing environment, Tamura and Yamada proposed a software reliability model for fault data aggregation and risk rate changes. Considering the changes in components, cloud computing, and the number of users, Tamura and Yamada proposed a software reliability model that integrates stochastic differential equations and jump diffusion processes. Considering the close relationship between cloud computing and big data system software through the network, Tamura et al. proposed a software reliability evaluation method for large-capacity cloud computing environments based on fault data clustering and neural networks.

[0005] The above software reliability model is established in the context of cloud computing, fully considering the relationship between cloud computing storage and big data communication, and establishing a corresponding open source big data system software reliability model, but there is a problem of complex modeling environment. Therefore, the present invention adopts a direct modeling method to establish a software reliability model for the actual development and testing environment of open source big data system software. Summary of the invention

[0006] By studying the fault data set of open source big data system software, it is found that the number of faults detected in each version of the open source big data system software in the later stage of software development and testing increases significantly. Therefore, this paper proposes a modeling method for the reliability model of open source big data system software based on Weibull-Weibull distribution, and experiments verify the accuracy and effectiveness of the proposed model.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] The present invention provides an open source big data system software reliability modeling method based on Weibull-Weibull distribution, comprising the following steps:

[0009] Step 1, make the following assumptions:

[0010] (1) The fault detection process of open source big data system software follows a non-homogeneous Poisson process;

[0011] (2) The faults found should be eliminated immediately and new faults should not be introduced;

[0012] (3) The faults detected in the open source big data system software are related to the remaining faults in the software;

[0013] (4) The fault detection rate of open source big data system software follows the Weibull-Weibull distribution;

[0014] Step 2: Based on assumption (1), the fault behavior detected by the open source big data system software is regarded as a counting process, represented by N(t), and the formula is as follows:

[0015]

[0016] Among them, m(t) represents the mean function, that is, the cumulative number of faults detected at time t, and n represents the actual number of faults detected;

[0017] Reliability is defined as follows:

[0018] R(μ / t)=exp[-(m(t+μ)-m(t))](2)

[0019] Among them, μ represents a period of time;

[0020] Step 3, based on assumption (3), derive the following equation:

[0021]

[0022] Where b(t) represents the risk rate function, and a represents the total number of faults initially present in the software;

[0023] Step 4, the hazard rate function is given by:

[0024]

[0025] Where b, d and e represent shape parameters, c represents scale parameter, F(x) represents cumulative distribution function, and f(x) represents probability density function;

[0026] Step 5: Based on assumption (4), substitute equation (3) into equation (4) to obtain the mathematical expression of the proposed model:

[0027] m(t)=a(1-exp(-b(exp(ct d )-1) e ))(6).

[0028] Furthermore, the least squares method is used to estimate the parameter estimates of the proposed model.

[0029] Furthermore, the least square method is expressed as:

[0030]

[0031] Among them, O(t j ) represents the time t j The number of faults actually detected at the time, j = 0, 1, 2, 3, ..., n;

[0032] Taking partial derivatives on both sides of equation (7),

[0033]

[0034] Solving the above equations can obtain the estimated values ​​of the model parameters.

[0035] Compared with the prior art, the present invention has the following advantages:

[0036] 1. For the first time, a software reliability model for open source big data system software considering Weibull-Weibull distribution is proposed.

[0037] 2. There is a significant increase in the number of faults discovered in the later stages of open source big data system development and software testing.

[0038] 3. Fault detection of open source big data system software follows the Weibull-Weibull distribution.

[0039] 4. Fitting and prediction performance experiments were conducted using six fault data sets and eight software reliability models. The experimental results show that the proposed model has better fitting and prediction performance than other software reliability models. The model can accurately and effectively predict the number of remaining faults in the open source big data system software testing process. 95% confidence interval and parameter sensitivity analysis were also performed. The results show that each parameter of the proposed model is an important parameter, and the proposed model has stable performance. The parameters of the model play an important role in the fault detection process of the open source big data system software. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 The cumulative number of faults detected for open source big data system software over time;

[0041] Figure 2 is the Weibull-Weibull probability density function;

[0042] Figure 3 To compare the fitting performance of software reliability models using 100% of the fault data sets (DS1 and DS2), (a), (b), (c), (d), (e), and (f) represent the comparison of the fitting performance of software reliability models using 100% of DS1-1, DS1-2, DS1-3 and DS2-1, DS2-2, DS2-3;

[0043] Figure 4 To compare the prediction performance of software reliability models using 90% of the fault data sets (DS1 and DS2), (a), (b), (c), (d), (e) and (f) represent the prediction performance comparison of software reliability models using 90% of DS1-1, DS1-2, DS1-3, DS2-1, DS2-2 and DS2-3. DETAILED DESCRIPTION

[0044] In order to further illustrate the technical solution of the present invention, the present invention is further described below through embodiments.

[0045] In this embodiment, two questions are discussed first: 1) Why do the faults detected in the later stages of software development and testing of various versions of open source big data system software increase significantly? 2) Why is a Weibull-Weibull distribution needed to simulate the fault detection process of open source big data system software?

[0046] A. Reasons for the increase in faults found in the later stages of open source big data system software development and testing

[0047] During the testing of open source big data system software, if more and more faults are found later, the following reasons may be the cause:

[0048] (1) Expanding the scope of testing: In the early stages of testing, the main focus may be on the core functions and critical paths of the system. However, as testing progresses, the scope of testing will gradually expand to cover more functions, boundary conditions, and abnormal situations. This may lead to more faults being discovered in later tests.

[0049] (2) Richness of test data: As testing progresses, the test team will prepare richer and more complex test data to simulate real data situations. These more complex data may expose previously undiscovered faults.

[0050] (3) Concurrency and load testing: In the later stages of testing, concurrency and load testing is usually performed to simulate multiple users accessing the system at the same time. This testing method can expose performance bottlenecks and failures under high load and concurrency conditions.

[0051] (4) Integration testing with other systems: Big data systems often need to be integrated with other systems, such as databases, message queues, caches, etc. In the later stages of testing, integration testing with other systems may expose previously undiscovered compatibility and interaction issues.

[0052] (5) Improvement of testing strategies and methods: As testing progresses, the testing team may continuously improve testing strategies and methods, such as adding automated testing, introducing new testing tools, etc. These improvements may make testing more comprehensive and in-depth, thereby discovering more faults.

[0053] In addition, the changes in the faults detected in the open source big data system components (i.e., versions 0.23.5, 0.23.6, and 0.23.7 ​​of YARN and versions 3.1.1, 3.1.2, and 3.1.3 of HDFS) are analyzed. Figure 1 In the figure, the horizontal axis represents time and the vertical axis represents the cumulative number of faults detected. Figure 1 It can be clearly seen that the number of faults detected in the open source big data system software in the later stages of project component testing has increased significantly.

[0054] B. Characteristics of the Weibull-Weibull distribution

[0055] The Weibull-Weibull distribution was proposed by Bourguignon et al. in 2014. The cumulative distribution function can be expressed as follows:

[0056] F(x)=1-exp{-b[exp(cx d )-1] e},x,b,c,d,e>0(1)

[0057] Where b, d and e represent shape parameters, and c represents the scale parameter.

[0058] The probability density function of the Weibull-Weibull distribution can be written as follows:

[0059] f(x)=bcdex d-1 [exp(cx d )-1] e-1 exp{-b[exp(cx d )-1] e +cx d}(2)

[0060] from Figure 2 As can be seen in , by changing the function parameters, the Weibull-Weibull probability density function has various curve shapes. For example, in Figure 2 In the example, its change decreases over time, or increases first and then decreases at different time points. The flexible changes of these curves or shapes play an important role in modeling the complex process of software fault detection in open source big data systems.

[0061] On the other hand, the hazard rate function can be given by:

[0062]

[0063] The hazard rate function of the Weibull-Weibull distribution has a variety of curves or shape changes over time, such as a decrease, increase, or bathtub shape of the fault detection rate. The various changes in the hazard rate function lay a solid foundation for simulating the fault detection process of open source big data system software. The present invention also uses multiple fault data sets to verify that the fault detection process of the big data system software follows the Weibull-Weibull distribution.

[0064] Based on the above content, the open source big data system software reliability modeling method based on Weibull-Weibull distribution includes the following steps:

[0065] Step 1, make the following assumptions:

[0066] (1) The fault detection process of open source big data system software follows a non-homogeneous Poisson process (NHPP);

[0067] (2) The faults found should be eliminated immediately and new faults should not be introduced;

[0068] (3) The faults detected in the open source big data system software are related to the remaining faults in the software;

[0069] (4) The fault detection rate of open source big data system software follows the Weibull-Weibull distribution;

[0070] Step 2: Based on assumption (1), the fault behavior detected by the open source big data system software is regarded as a counting process, represented by N(t), and the formula is as follows:

[0071]

[0072] Among them, m(t) represents the mean function, that is, the cumulative number of faults detected at time t, and n represents the actual number of faults detected;

[0073] Reliability is defined as follows:

[0074] R(μ / t)=exp[-(m(t+μ)-m(t))](5)

[0075] Among them, μ represents a period of time;

[0076] Step 3, based on assumption (3), derive the following equation:

[0077]

[0078] Where b(t) represents the risk rate function, and a represents the total number of faults initially present in the software;

[0079] Step 4: Based on assumption (4), substitute equation (6) into equation (3) to obtain the mathematical expression of the proposed model:

[0080] m(t)=a(1-exp(-b(exp(ct d )-1) e ))(7).

[0081] The estimation method of the obtained model parameters uses the least squares method to estimate the parameter values ​​of the model. The least squares estimation (LSE) method can be expressed as:

[0082]

[0083] Among them, O(t j ) indicates time tj The number of faults actually detected at time , j = 0, 1, 2, 3, ..., n.

[0084] Taking partial derivatives on both sides of equation (8),

[0085]

[0086] Solving the above equations can obtain the estimated values ​​of the model parameters.

[0087] Example 2

[0088] In this embodiment, the collected fault datasets, the models used for comparison, and the model comparison criteria are introduced. The performance of the model using two fault datasets from open source big data system components is also discussed, each fault dataset includes three sub-datasets collected from continuously released open source big data system components.

[0089] A. Fault Dataset

[0090] Two failure datasets collected from two different open source big data system components, namely YARN and HDFS, are used in the issue tracking system of https: / / issues.apache.org / . Three sub-datasets are collected from YARN, such as YARN 0.23.5, YARN 0.23.6, and YARN 0.23.7. Three sub-datasets are collected from HDFS, such as HDFS 3.1.1, HDFS 3.1.2, and HDFS 3.1.3.

[0091] The first failure dataset (DS1-1) for YARN 0.23.5 includes 28 failures detected in 29 weeks from May 3, 2012 to November 16, 2012. The second failure dataset (DS1-2) for YARN 0.23.6 includes 15 failures and 25 failures detected in 48 weeks from February 22, 2012 to January 23, 2013. In the third failure dataset (DS1-3) for YARN 0.23.7, 25 failures were detected in 18 months from October 7, 2011 to March 12, 2013. In the fourth failure dataset (DS2-1) for HDFS 3.1.1, 84 failures were detected in 47 months from September 19, 2014 to July 25, 2018. In the fifth failure dataset (DS2-2) of HDFS 3.1.2, 65 failures were detected in 35 months from March 31, 2016 to January 24, 2019. In the sixth failure dataset (DS2-3) of HDFS 3.1.3, 69 failures were detected in 54 months from April 17, 2015 to October 2, 2019.

[0092] It should be noted that the fault types in the collected open source big data system components include Bug, Task, and Sub-task, but do not include Improvement, New Feature, Test, and Wish.

[0093] Here are the reasons:

[0094] (1) Improvement: refers only to the improvement or enhancement of existing functions or tasks, rather than reporting system failures or problems.

[0095] (2) New Feature: refers to a function or feature that is planned to be implemented in a future version of the software without reporting an existing system failure.

[0096] (3) Test: Usually used to track software testing related tasks, test cases, and test plans. This may include writing, executing, and managing test cases, rather than reporting system failures.

[0097] (4) Wish: Usually used to express a wish or suggestion for system improvements or new features, rather than reporting a malfunction or problem with an existing system.

[0098] B. Comparison Models

[0099] In order to make a thorough comparison of the performance of the models, seven software reliability models are used to compare the fitting and prediction performance of six fault datasets of open source big data system software. As can be seen from Table 1, the seven software reliability models include Goel-Okumoto (GO) model, Delay S-shaped (DSS) model, Inflection S-shaped (ISS) model, Generalized GO (GGO) model, Zhang-Teng-Pham model, Li model and the proposed model. Among these software reliability models, closed-source software reliability models include GO, DSS, ISS, GGO and Zhang-Teng-Pham model. Open-source software reliability models include Li model and the proposed model. Perfectly debugged software reliability models include GO, DSS, ISS, GGO, Li model and the proposed model. Zhang-Teng-Pham model is an imperfectly debugged software reliability model.

[0100] C. Model comparison criteria

[0101] Using MSE, R 2 , TS, MEOP and Variance to evaluate the performance of the model. They can be expressed as follows:

[0102]

[0103] Where, represents the time t j The expected cumulative number of faults detected at t j ), indicating that at time t j The actual number of failures observed when n and θ represent the sample size and the number of model parameters, respectively.

[0104]

[0105] In formula (11), m(t j ), and n are the same as in formula (10).

[0106] Note that the smaller the MSE, TS, MEOP, and Variance values, the better the model performance. 2 The larger the value, the better the model fitting performance.

[0107] In order to properly evaluate and compare the performance of various attributes of SRMS, the ranking (NCD k ) method. It can be expressed as follows:

[0108]

[0109] Where n1 and n2 represent the number of model comparison criteria and compared software reliability models, respectively, and χ kj represents the kth software reliability model and the standard value of the jth standard, ψ j represents the weight when the criterion index is j, NCD k Smaller values ​​indicate better model performance.

[0110] Table 1 Various software reliability growth models

[0111]

[0112] D. Results and Discussion of Model Performance Comparison

[0113] (1) Comparison of model fitting performance

[0114] In order to verify the accuracy and effectiveness of the proposed model, the fitting performance was compared with six fault data sets and seven software reliability models. Table 2 lists the parameter estimation values ​​of the model proposed in the present invention using the least squares method with 100% and 90% of the fault data sets of DS1 and DS2, respectively. Table 3 shows that the proposed model has excellent fitting performance compared with other models. For example, in Table 3, using 100% of DS2-1, the MSE, TS, and MEOP of the ISS model are approximately 4.25, 1.35, and 0.95 times greater than those of the model proposed in the present invention, respectively. Using 100% of DS2-2, the MSE, TS, and MEOP of the ISS model are nearly 4.43, 1.41, and 0.89 times greater than those of the model proposed in the present invention, respectively. From Table 5, the ranking results of the software reliability models can be clearly seen. The model proposed in the present invention ranks first, the ISS model ranks second, and the GO model ranks worst. The ranking order of other software reliability models is not fixed. These results show that the model proposed in the present invention has better fitting performance than other software reliability models. These results can be seen from Figure 3 Clearly seen in. Figure 3 A comparison of the fitting performance of eight software reliability models using six fault datasets is shown. Figure 3 It can be seen that the fitting performance of the proposed model is better than other software reliability models.

[0115] (2) Comparison of model prediction performance

[0116] In order to verify the prediction performance of the proposed model, corresponding experiments were carried out. It can be seen from Table 4 that, except for DS2-2 using 90%, the prediction performance of the model proposed by the present invention is better than other software reliability models using 90% fault data sets. For example, in Table 4, using 90% of DS1-3, the MSE, TS, Variance and MEOP of the ISS model are approximately 3.58, 1.89, 1.9 and 1.4 times that of the model proposed by the present invention. Using 90% of DS2-1, the MSE, TS, Variance and MEOP of the GGO model are approximately 7.19, 2.68, 2.52 and 3.18 times that of the model proposed by the present invention. It can be seen from Table 6 that the model proposed by the present invention, the ISS model and the GO model rank first, second and last respectively. In Table 4, using 90% of DS1-1, the variance value (0.4818) of the ISS model is smaller than the variance value (0.5248) of the proposed model. However, it can be seen from Table 6 that, through comprehensive comparison, the prediction performance of the model proposed in the present invention is better than that of the ISS model. In addition, in Table 4, using 90% of DS2-2, the Variance and MEOP values ​​of the ISS model are smaller than those of the model proposed in the present invention. The values ​​of the ISS model are 2.1273 and 0.6264, and the values ​​of the model proposed in the present invention are 2.3336 and 0.6564. Table 6 shows that when using 90% of DS2-2, the ISS model ranks first and the model proposed in the present invention ranks second. However, the ranking value of the proposed model is close to that of the ISS model.

[0117] The prediction performance of the model proposed in this invention is better than other models. It can be seen from Table 6 that, except for the model proposed in this invention, the prediction performance of other models is not stable. For example, using 90% of DS2-1, the ISS model ranks third, while using 90% of DS2-2 ranks first, and other cases rank second. Other software reliability models have the same changes as the ISS model. Note that it can be seen from Tables 5 and 6 that the GO model is the worst regardless of using 100% or 90% of the fault data set. This shows that the GO model is not suitable for reliability evaluation of open source big data system software. From Figure 4 It can be seen that the model proposed in the present invention is the best among all software reliability models in terms of prediction performance using 90% of the fault data set.

[0118] In summary, the model proposed in this paper has better fitting and prediction performance in the fault detection process of open source big data system software. The proposed model predicts the number of remaining faults in the software most accurately and has the most stable performance. The experiment verifies that the fault detection of open source big data system software follows the Weibull-Weibull distribution, and verifies the effectiveness of the proposed model in fault prediction during the testing process of open source big data system software.

[0119] Table 2 Parameter estimates of the proposed model

[0120]

[0121] Table 3 Comparison of fitting performance of software reliability model using big data system software failure dataset

[0122]

[0123]

[0124] Table 4 Comparison of prediction performance of software reliability models using big data system software failure dataset

[0125]

[0126]

[0127] Table 5 Comparison and ranking of fitting performance

[0128]

[0129] Table 6 Prediction performance comparison ranking

[0130]

[0131]

Claims

1. A reliability modeling method for open source big data system software based on Weibull-Weibull distribution, characterized by: The following steps are involved: Step 1, make the following assumptions: (1) The fault detection process of open source big data system software follows a non-homogeneous Poisson process; (2) The faults found should be eliminated immediately and new faults should not be introduced; (3) The faults detected in the open source big data system software are related to the remaining faults in the software; (4) The fault detection rate of open source big data system software follows the Weibull-Weibull distribution; Step 2: Based on assumption (1), the fault behavior detected by the open source big data system software is regarded as a counting process, represented by N(t), and the formula is as follows: Among them, m(t) represents the mean function, that is, the cumulative number of faults detected at time t, and n represents the actual number of faults detected; Reliability is defined as follows: R(μ / t)=exp[-(m(t+μ)-m(t))](2) Among them, μ represents a period of time; Step 3, based on assumption (3), derive the following equation: Where b(t) represents the risk rate function, and a represents the total number of faults initially present in the software; Step 4, the hazard rate function is given by: Where b, d and e represent shape parameters, c represents scale parameter, F(x) represents cumulative distribution function, and f(x) represents probability density function; Step 5: Based on assumption (4), substitute equation (3) into equation (4) to obtain the mathematical expression of the proposed model: m(t)=a(1-exp(-b(exp(ct d )-1) e ))(6)。 2. The open source big data system software reliability modeling method based on Weibull-Weibull distribution according to claim 1, characterized in that: The parameter estimates of the proposed model were estimated using the least squares method.

3. The open source big data system software reliability modeling method based on Weibull-Weibull distribution according to claim 2, characterized in that: The least squares method is expressed as: Among them, O(t j ) represents the time t j The number of faults actually detected at the time, j = 0, 1, 2, 3, ..., n; Taking partial derivatives on both sides of equation (7), Solving the above equations can obtain the parameter estimates of the model.