A Bayesian verification method, system and device for reliability of discrete software

Abstract

The present application belongs to the field of software analysis, and specifically discloses a Bayesian verification method, system and device for reliability of discrete software, the method comprising: obtaining a discrete data set for reliability growth testing of discrete software; determining a reliability growth model of discrete software by combining the data set and a non-homogeneous Poisson process in a discrete time domain; estimating the probability of discrete software successfully executing a preset specified task again based on the reliability growth model; setting the prior distribution of reliability to a beta distribution, and estimating the parameters of the beta distribution based on the probability of successfully executing a preset specified task and the maximum entropy method; determining the posterior distribution of reliability based on the prior distribution and Bayes' theorem; determining the number of test cases required for reliability verification of discrete software under a preset tolerable number of failures based on the posterior distribution; and performing reliability verification based on the number of test cases and the tolerable number of failures. Through the present application, an effective reliability verification scheme for discrete software is proposed.

Description

Technical Field

[0001] The present application belongs to the field of software analysis, and more specifically, to a Bayesian verification method, system and device for reliability of discrete software. Background Art

[0002] With the rapid development of the intelligent network era, the application of software in human life is becoming more and more extensive and in-depth, and system crashes and paralysis caused by software failures occur frequently. Reliability is an important indicator in the software quality system. In order to ensure the safety and reliability of the software put into use, many software subscribers, especially those in safety-critical fields, will specify reliability indicators at the beginning. How to formulate a reliability verification plan in the final product acceptance stage and objectively and reliably verify whether the software meets the specified reliability indicator requirements is an important issue in software quality work.

[0003] At present, a series of software reliability verification test schemes such as life cycle test, probability ratio sequential test, Bayesian verification test, etc. have been proposed by referring to the hardware reliability verification test standards at home and abroad. According to the specified reliability indicator type, they can be divided into two categories: discrete schemes (reliability indicators are success rate or failure rate) and continuous schemes (reliability indicators are failure rate, mean time before failure or mean time between failures). However, most of the existing verification schemes are suitable for continuous software. Kapur pointed out that because it is more convenient to record failure data in discrete time (such as every hour / day / week), many software failure data are discrete data, so it is more in line with objective reality to establish a discrete time domain model, but due to the difficulty of mathematical processing, there are few research results on discrete models. Cai Kaiyuan, Kumaran and others have long called for strengthening the research on the reliability of discrete software, because many software such as missile launch software and control switch software need to measure their reliability by the success or failure of a task.

[0004] Since software reliability indicators in safety-critical areas are often set very high, reliability verification requires a large number of test cases. The Bayesian verification method uses two parts of information, product prior data and field verification test data, to give a comprehensive verification conclusion. It can reduce the number of verification test cases and improve the problem of long reliability verification period. Therefore, it is the most fruitful method in discrete software reliability verification research. In the Bayesian method, the determination of prior distribution is the cornerstone. However, discrete software accumulates discrete reliability growth test data before delivery, and the prior distribution determination method of the existing Bayesian solution is still ideal and subjective in processing this prior information. For example, the method of determining the prior distribution without prior information assumes that the software success rate obeys a uniform distribution between 0 and 1. However, the software success probability at the end of the growth test should have increased to a certain value and will not obey a uniform distribution. Wu constructed a decreasing function model in the continuous time domain as the prior distribution of the software failure rate, and then converted the discrete growth test samples into continuous samples through sample conversion to solve the model. This method of determining the prior distribution reflects the characteristics of reliability growth, but the parameter setting of the provided sample conversion method is too subjective, and the calculation results vary greatly. The dynamic prior information Bayesian integration method determines the prior distribution of the software success rate as a Beta distribution, and uses the moment estimation method to estimate the two parameter values ​​of the Beta distribution, making full use of the discrete prior information, but the parameter estimation method is not reasonable enough for the processing of growth test data. Summary of the invention

[0005] In view of the defects of the prior art, the purpose of this application is to provide a Bayesian verification method, system and device for the reliability of discrete software, aiming to solve the problem that the existing reliability verification methods for discrete software are not effective.

[0006] To achieve the above objectives, in a first aspect, the present application provides a Bayesian reliability verification method for discrete software, wherein the discrete software records data of task execution failures in discrete time; comprising:

[0007] Obtain a discrete data set for performing reliability growth test on the discrete software; the discrete data set includes: the number of test cases executed by each group of tests and the number of successfully executed test cases; determine a discrete software reliability growth model by combining the discrete data set and a non-homogeneous Poisson process in a discrete time domain; and estimate the probability of the discrete software successfully executing a preset specified task again after the reliability growth test based on the discrete software reliability growth model;

[0008] The prior distribution of the reliability of the discrete software in the reliability verification test phase is set to beta distribution, and the parameters of the beta distribution are estimated based on the probability of successfully executing the preset specified task again and the maximum entropy method; the reliability is the probability of the discrete software successfully executing the specified task in the reliability verification test phase;

[0009] Determine the posterior distribution of the reliability based on the prior distribution after the estimated parameters and Bayes' theorem; determine the number of test cases required for reliability verification testing of the discrete software under a preset tolerable number of failures according to the posterior distribution;

[0010] The discrete software is subjected to reliability verification testing according to the number of test cases and the preset tolerable number of failures.

[0011] It should be noted that, unlike continuous software such as safety-critical software, discrete software usually records failure data at discrete time (e.g., every hour / day / week) for greater convenience, and its failure data is mostly discrete data, so establishing a discrete time domain model is more in line with objective reality. The above-mentioned discrete software is, for example, launch control software, airborne escape software, shipboard escape software, or control switch software.

[0012] This application establishes a reliability growth model based on the reliability growth test of discrete software. As the test grows, the defects of the discrete software are gradually detected, and the number of defects contained therein will gradually decrease. Therefore, the probability of passing the corresponding reliability verification will also gradually increase, avoiding the situation where the discrete model established by the prior art is too ideal. This application combines the defect detection process of discrete software in the discrete time domain to establish a corresponding reliability growth model for it, which is more compatible with the detection process of discrete software; then, the reliability growth test information of discrete software is used as prior information, and the prior distribution obtained by using the discrete software reliability growth model and the maximum entropy method is also more accurate; because the maximum entropy method can be closer to the actual situation of the reliability of the reliability growth model gradually improving, it can greatly avoid the influence of subjective factors, and provide a more realistic and objective method for determining the prior distribution of the success rate of discrete software, so as to propose a Bayesian verification scheme for software reliability designed from the perspective of protecting the interests of users.

[0013] In a possible implementation, determining a discrete software reliability growth model includes:

[0014] A counting function in a discrete time domain is used to represent the number of defects detected in the executed test cases in the reliability growth test; the number of detected defects is the difference between the number of executed test cases and the number of successfully executed test cases;

[0015] The number of failed use cases is equal to the number of detected defects, that is, each time a use case fails to execute, a defect is detected.

[0016] In combination with the counting function in the discrete time domain, a non-homogeneous Poisson process in the discrete time domain is used to represent the defect detection function in the reliability growth test; the defect detection function is related to the cumulative defect function, and the cumulative defect function is the expected function of the counting function;

[0017] A discrete software reliability growth model is determined in combination with the defect detection function; wherein the discrete software reliability growth model is a reliability function of discrete software; the reliability function is the probability that after the discrete software has cumulatively executed multiple test cases, no defects will be detected after executing at least one test case; each defect will cause the discrete software to fail once it is triggered.

[0018] In a possible implementation, the reliability growth model includes two parameters: the total initial defect count and the defect detection rate. The two parameters are estimated based on the discrete data set using the maximum likelihood estimation method; the reliability function and the cumulative defect function are both related to the two parameters.

[0019] In a possible implementation, the defect detection function is:

[0020]

[0021]

[0022] Wherein, P{N(n)=k} represents the probability of detecting k defects when the discrete software executes n test cases; D(n) represents the cumulative defect function of the cumulative execution of n test cases, and D(n) is the expected function of N(n); N(n) is a counting function in the discrete time domain, which represents the number of defects detected in the n test cases that have been executed; exp[] represents the natural exponential function; a represents the total number of initial defects, and b represents the defect detection rate; R(h|n) is the reliability function of the discrete software, which refers to the probability that no defects will be found when the discrete software executes h test cases after the discrete software has cumulatively executed n test cases.

[0023] In a possible implementation, the two parameters are estimated using a maximum likelihood estimation method, including:

[0024]

[0025] Wherein, the reliability growth test includes l group tests, (n i ,x i ) indicates that the i-th group of tests is executed n i There are x test cases i The number of test cases executed successfully and the number of defects detected is n i -x i ;(n i-1,x i-1 ) indicates that the i-1th group of tests is executed n i-1 There are x test cases i-1 Use cases were successfully executed; (n l ,x l ) indicates that the first group of tests is executed l There are x test cases l The use cases are successfully executed, 1≤i≤l; the data set Ω={(n1,x1),(n2,x2),...,(n l ,x l )}; a represents the total number of initial defects, and b represents the defect detection rate.

[0026] In a possible implementation, the discrete software reliability growth model is:

[0027] R(h|n)=exp{-[D(n+h)-D(n)]}

[0028] Wherein, R(h|n) is the reliability function of discrete software; exp{} represents the natural exponential function; D(n) represents the cumulative defect function of cumulatively executing n test cases, and D(n+h) represents the cumulative defect function of cumulatively executing n+h test cases; the reliability growth test includes l groups of tests, and the i-th group of tests executes n i test cases, when n is the sum of all test cases executed in the reliability growth test 1≤i≤l; when h is 1, It represents the probability that the discrete software can successfully execute the specified task again after the reliability growth test.

[0029] In a possible implementation, the prior distribution π(R) is:

[0030]

[0031] Among them, Be(u,v) is the Beta distribution, B(u,v) is the Beta function, u and v are the hyperparameters of the prior distribution; R represents the reliability of discrete software in the reliability verification stage, which is a random variable;

[0032] The hyperparameters of the Beta distribution are determined using the maximum entropy method using the following system of two-variable equations:

[0033]

[0034] Among them, H B (u,v) is the information entropy function of the Beta distribution;

[0035] Solve the above binary equation system. If the estimated numerical solution of the hyperparameter is (u0, v0), then the prior distribution π(R) after estimating the parameters is Be(u0, v0).

[0036] In a possible implementation, the posterior distribution π(R|N, S) is as follows:

[0037] π(R|N, S) = Be(u0 + S, v0 + N - S);

[0038] where N is the number of test cases executed in the reliability verification phase, and S is the number of test cases that are successfully executed among the N test cases;

[0039] When the preset tolerable number of failures is c, determine the number of test cases through the following inequality:

[0040] P{R < R0|r = c, N} = Be(R0; u0 + N - c, v0 + c) ≤ γ

[0041] where P{R < R0|r ≤ c, N} is an increasing function of r, r represents the number of defects detected when the discrete software executes N test cases in the reliability verification phase; γ is the maximum risk that the user can bear; R0 represents the lower limit of the reliability that the user can accept;

[0042] Take the minimum N when the above inequality holds as the number of test cases.

[0043] In a second aspect, the present application provides a reliability Bayesian verification system for discrete software, where the discrete software records data on task execution failures at discrete times; the system includes:

[0044] An increasing test data set acquisition module, configured to acquire a discrete data set for reliability growth testing of the discrete software; the discrete data set includes: the number of test cases executed in each group of tests and the number of cases that are successfully executed; combine the discrete data set and a non - homogeneous Poisson process in the discrete time domain to determine a reliability growth model for the discrete software; and based on the reliability growth model of the discrete software, estimate the probability that the discrete software will successfully execute a preset specified task after the reliability growth test;

[0045] A prior distribution determination module, configured to set the prior distribution of the reliability of the discrete software in the reliability verification test phase as a beta distribution, and estimate the parameters of the beta distribution based on the probability of successfully executing the preset specified task again and the maximum entropy method; the reliability is the probability that the discrete software successfully executes the specified task in the reliability verification test phase;

[0046] A posterior distribution determination module, used to determine the posterior distribution of the reliability based on the prior distribution after the estimated parameters and Bayes' theorem; determine the number of test cases required for reliability verification testing of the discrete software under a preset tolerable number of failures according to the posterior distribution;

[0047] The reliability verification test module is used to perform reliability verification test on the discrete software according to the number of test cases and the preset tolerable number of failures.

[0048] In a third aspect, the present application provides an electronic device comprising: at least one memory for storing programs; and at least one processor for executing the programs stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method described in the first aspect or any possible implementation of the first aspect.

[0049] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.

[0050] In a fifth aspect, the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.

[0051] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0052] In general, the above technical solutions conceived by the present application have at least the following beneficial effects compared with the prior art:

[0053] The present application provides a Bayesian verification method, system and equipment for the reliability of discrete software. In view of the problem that the processing method of the prior information of the reliability growth test of discrete software by the existing Bayesian verification method of discrete software reliability is too ideal and subjective, a method for determining the prior distribution of software success rate is given by using the discrete software reliability growth model and the principle of information entropy. On this basis, a Bayesian reliability verification scheme based on maximum a posteriori risk is designed from the perspective of protecting the interests of users. It has been verified by examples that the proposed scheme is more accurate in mining the prior information of software reliability growth test, can reduce the number of test cases under the premise of ensuring the credibility of the verification scheme, and has economic use value.

[0054] The present application provides a Bayesian verification method, system and equipment for the reliability of discrete software. Since the present application uses a discrete software reliability growth model to give a point estimate of the software success rate, it has a higher evaluation accuracy when the sample size is large, and software with high reliability index requirements usually undergoes high-intensity reliability growth tests and accumulates rich test data before delivery. Therefore, in terms of the applicability of the method, the Bayesian verification scheme proposed in the present application has good applicability for the reliability verification of software with high reliability index requirements such as software in safety-critical fields. Finally, most of the existing software reliability Bayesian verification schemes are suitable for continuous software. With the rapid development of the era of intelligent networking, discrete software is becoming more and more popular. For example, the switch control functions of many smart products such as smart homes, driverless cars, drones, and smart medical care are no longer completed by traditional mechanical switches, but by mobile phones or computers to remotely control. Therefore, the discrete software reliability verification method proposed in the present application can better meet the actual needs of engineering. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a flow chart of the Bayesian verification method for the reliability of discrete software provided in the embodiment of the present application;

[0056] Figure 2 is a comparison chart of the actual test data provided by the embodiment of the present application and the fitting data of the reliability growth model;

[0057] Figure 3 This is a diagram of the architecture of a Bayesian reliability verification system for discrete software provided in an embodiment of the present application;

[0058] Figure 4 It is an architectural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0060] The term "and / or" in this application is a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The symbol " / " in this application indicates that the associated objects are in an or relationship, for example, A / B means A or B.

[0061] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0062] Next, the technical solutions provided in the embodiments of the present application are introduced.

[0063] This application uses the reliability growth test information of discrete software as prior information, and uses the discrete software reliability growth model and the information entropy principle to give a method for determining the prior distribution of software success rate. Then, from the perspective of protecting the interests of users, a Bayesian verification scheme for software reliability is designed. Finally, the effectiveness of the proposed method is verified through examples.

[0064] Figure 1 A flowchart of a Bayesian reliability verification method for discrete software provided in an embodiment of the present application, such as Figure 1 As shown, the following steps are included:

[0065] Step S101, obtaining a discrete data set for performing reliability growth test on discrete software; the discrete data set includes: the number of test cases executed by each group of tests and the number of successfully executed test cases; combining the discrete data set and the non-homogeneous Poisson process in the discrete time domain, determining a discrete software reliability growth model; and estimating the probability of the discrete software successfully executing a preset specified task again after the reliability growth test based on the discrete software reliability growth model;

[0066] For example, determining a discrete software reliability growth model includes:

[0067] Step S1011, using a counting function in a discrete time domain to represent the number of defects detected in the executed test cases in the reliability growth test; the number of detected defects is the difference between the number of executed test cases and the number of successfully executed test cases;

[0068] Step S1012, combining the counting function in the discrete time domain, using a non-homogeneous Poisson process in the discrete time domain to represent the defect detection function in the reliability growth test; the defect detection function is related to the cumulative defect function, and the cumulative defect function is the expected function of the counting function;

[0069] Step S1013, determining a discrete software reliability growth model in combination with the defect detection function.

[0070] Among them, the discrete software reliability growth model is the reliability function of the discrete software; the reliability function is the probability that the discrete software will not detect defects after executing at least one test case after accumulating multiple test cases; each defect will cause the discrete software to fail once it is triggered.

[0071] Step S102, setting the prior distribution of the reliability of the discrete software in the reliability verification test phase to beta distribution, and estimating the parameters of the beta distribution based on the probability of successfully executing the preset specified task again and the maximum entropy method; the reliability is the probability of the discrete software successfully executing the specified task in the reliability verification test phase;

[0072] Step S103, determining the posterior distribution of the reliability based on the prior distribution after the estimated parameters and Bayes' theorem; determining the number of test cases required for reliability verification testing of the discrete software under a preset tolerable number of failures according to the posterior distribution;

[0073] Step S104: performing reliability verification test on the discrete software according to the number of test cases and the preset tolerable number of failures.

[0074] For example, the reliability verification test is as follows: according to the number of test cases N determined in step S104 and the preset tolerable number of failures c, the reliability verification test is carried out on the discrete software, the specified task is repeated N times independently, and the number of execution failures r is recorded. If r≤c, the discrete software passes the reliability verification and is qualified; otherwise, the software is rejected and the verification is unqualified.

[0075] It should be noted that software reliability verification usually includes two stages: reliability growth testing and reliability verification testing. Specifically, the software will undergo a series of routine testing activities during the R&D stage, such as unit testing, integration testing, system testing, etc. If there are clear reliability index requirements, the R&D party will also organize reliability growth testing after the completion of routine testing activities to improve the software reliability level by continuously finding and removing defects so that it meets the specified reliability index requirements. The test data accumulated at this stage can be used as prior information before software delivery and acceptance.

[0076] The reliability index of discrete software is generally the software success rate / failure rate. Its reliability growth test is usually carried out in the form of "testing, concentrated defect elimination, retesting, and concentrated defect removal". Between any two concentrated defect removals, only the number of detected defects is counted, and no defects are debugged. The software has l defects removed during the growth test period. The recorded test data set is Ω={(n1,x1),(n2,x2),...,(n l ,x l )} means, where (n i ,xi ) indicates that the i-th group of tests has executed n i There are x test cases i The test cases were successfully executed, n i -x i Assuming that each defect causes failure once triggered, {N(n), n≥0} also represents the number of defects found in n use cases.

[0077] A counting process {N(n), n≥0} in the discrete time domain is introduced to represent the number of failed test cases among the n test cases that have been executed. Under the assumption that defect detection is independent of each other and each defect causes failure once triggered, the defect detection process can be described broadly by the following non-homogeneous Poisson process (NHPP) in the discrete time domain as follows:

[0078]

[0079] Wherein, D(n) is the expected function of N(n), satisfying D(n)=E[N(n)], and represents the average number of defect detections.

[0080] Therefore, for a discrete software that has executed n test cases cumulatively, the probability that it will execute h test cases without failure is defined as the reliability of the discrete software. When n is given, according to formula (1), the reliability can be expressed by the conditional probability R(h|n) as follows:

[0081]

[0082] Among them, formula (2) is called the discrete software reliability growth model, and R(h|n) is uniquely determined by the expected function D(n). At present, many scholars have proposed different assumptions about the software testing environment, and thus proposed a variety of mathematical models for R(h|n) and D(n). The common ones are the geometric model, the constant defect detection rate model, and the

[17] , Cai model, discrete Gompertz curve model, etc. Under the assumption that each defect has the same probability of being detected and can be perfectly removed, the cumulative defect function D(n) and the reliability function R(h|n) are usually described by the constant defect detection rate model:

[0083]

[0084] Among them, a and b are two parameters of the growth model, a represents the total number of initial defects lurking in the software, and b represents the defect detection rate. The values ​​of a and b can be based on the obtained test data set Ω = {(n1,x1), (n2,x2), ..., (n l ,x l)} is estimated using the maximum likelihood estimation method, and the calculation formula is as follows:

[0085]

[0086] The numerical solution of (4) is the maximum likelihood estimate of a and b. Substituting the estimated values ​​of a and b into (3) can obtain the software reliability R(h|n) given the task duration h. In particular, when n is When h is 1, It means that a The probability that the software will successfully perform the specified task again after passing a reliability growth test case.

[0087] The accuracy of the success rate point estimate directly affects the effectiveness of the subsequent prior distribution determination method. For this reason, the mean square error (MSE) and the related index R-square can be used to measure the fitting effect of the model, where:

[0088]

[0089] in, Indicates that when n is executed i When there are test cases, the cumulative number of defects calculated based on the discrete software reliability growth model, the smaller the MSE value, the better the model fits the test data, and the closer the R-square value is to 1, the closer the model is to the actual operation of the software.

[0090] It is understandable that the size of the sample has an impact on This has an important impact on the effectiveness of the evaluation results. Since software with high reliability requirements usually undergoes high-intensity reliability growth testing before delivery, rich test data can ensure the robustness and effectiveness of the evaluation results. Therefore, for software with high reliability requirements such as safety-critical software, this estimate has a higher evaluation accuracy.

[0091] Furthermore, assume that the software executes N test cases in the reliability verification phase, and the number of successfully executed test cases is S. Since there is no defect removal in the reliability verification test, the reliability verification test is regarded as an N-fold Bernoulli test, and the random variable S obeys the binomial distribution B(N,R), and the likelihood function is:

[0092]

[0093] Among them, R is the software reliability level of delivery verification. It represents the number of combinations of S test cases that are successfully executed in the N test cases executed in the reliability verification phase. If the parameter R is regarded as a random variable, R has a prior distribution and a posterior distribution. Let the prior distribution of R be π(R). For convenience, the conjugate distribution method is used to determine the prior distribution as Be(u,v) distribution, then:

[0094]

[0095] in, is the Beta function, u and v are the hyperparameters of the prior distribution, and their values ​​are to be determined.

[0096] The estimation of hyperparameters is actually the processing of prior information. In the absence of other prior information except for the reliability growth test information, in order to reduce the influence of other subjective factors, the maximum entropy method is used to determine the hyperparameters of the Beta distribution and solve the following equations:

[0097]

[0098] Among them, H B (u,v) is the information entropy function of Beta distribution, is the point estimate of the success rate given by (3). ER represents the expectation of the reliability function R(h|n).

[0099] Since the Beta distribution entropy function H B The expression of (u,v) is complicated, so we simplify it first:

[0100]

[0101] because Introducing the Psi function Then we have:

[0102]

[0103] Where Γ(x) represents the gamma function with independent variable x;

[0104] Then the entropy function H B (u,v) can be further simplified as:

[0105] H B (u,v)=lnB(u,v)-(u-1)[Ψ(u)-Ψ(u+v)]-(v-1)[Ψ(v)-Ψ(u+v)]

[0106] =lnB(u,v)-(u-1)Ψ(u)-(v-1)Ψ(v)+(u+v-2)Ψ(u+v)

[0107] So we can get the equivalent two-variable equation system:

[0108]

[0109] Find the numerical solution \((u_0, v_0)\) of the above binary equation system, and the prior distribution of \(R\) is determined as:

[0110] \(\pi(R)=Be(u_0, v_0)\ (11)\)

[0111] In an alternative example, the embodiment of the present application provides a Bayesian verification scheme for discrete software reliability. This scheme is a verification scheme based on the maximum posterior risk. Suppose the software executes \(N\) test cases during the reliability verification test phase, and \(S\) cases are executed successfully. Given that the prior distribution of \(R\) is determined, the posterior distribution of \(R\) can be obtained by Bayes' theorem as:

[0112]

[0113] Based on this, a Bayesian verification scheme for software reliability is designed.

[0114] Specifically, once the software passes the verification, if it fails due to unqualified actual quality level, the user bears a posterior risk \(P\{R < R_0|r \leq c, N\}\) at this time. Design a single-risk reliability verification scheme from the perspective of protecting the interests of the user, that is, it is necessary to satisfy:

[0115] \(P\{R < R_0|r \leq c, N\} \leq \gamma\ (13)\)

[0116] Among them, \(\gamma\) is the maximum risk that the user can bear, and \(R_0\) is the reliability index specified by the user, indicating the lower limit of the software success rate acceptable to the user. Since during the specified verification test period, the software can pass the verification when the number of software failures is any value from 0 to \(c\), so equation (13) includes \(c + 1\) cases. To protect the interests of the user to the greatest extent, control the maximum value of the \(c + 1\) risks, then it is required that:

[0117] \(\max\{P(R < R_0|r = 0, N), P(R < R_0|r = 1, N), \cdots, P(R < R_0|r = c, N)\} \leq \gamma\ (14)\)

[0118] It can be proved that \(P\{R < R_0|r\}\) is an increasing function of \(r\). Then, given the tolerable number of failures \(c\), make the following inequality:

[0119]

[0120] The smallest \(N\) that makes the inequality hold is the total number of test cases required for the reliability verification test.

[0121] It should be noted that Be(R0;u0+Nc,v0+c) in formula (15) is the specific case where S is Nc in the posterior distribution Be(u0+S,v0+NS) obtained previously.

[0122] The meaning of formula (15) is: if the software passes the verification according to the (N,c) scheme, then when the software is delivered to the user, the probability that the software successfully completes a task is not less than R0 and the credibility is not less than 1-γ. This ensures the software quality and user interests. In particular, when c is 0, the software adopts a no-failure verification scheme, that is, no failure should be found during the specified reliability verification test period, and once a failure occurs, the software is rejected. For software with high reliability requirements, such as software used in safety-critical fields, a no-failure verification scheme can be adopted.

[0123] Furthermore, assuming that the reliability index of a discrete software is (R0,γ), where R 0 is the minimum success rate acceptable to the user, and γ is the maximum risk that the user can bear. According to the solution proposed in this application, software reliability verification can be implemented in the following steps:

[0124] Step 1: Point estimation of software success rate and validity test of estimation. According to the failure data generated by the software during the reliability growth test period, Ω={(n1,x1),(n2,x2),...,(n l ,x l )}, use formula (1) to estimate the probability that the software successfully completes a specified task The parameters a and b of formula (1) are obtained by maximum likelihood estimation, and the validity of the point estimate is tested by mean square error (MSE) and correlation index (R-square).

[0125] Step 2: Develop a Bayesian reliability verification scheme. Combined with formula (7), the hyperparameters u0, v0 of the Beta distribution are solved, and formula (12) is used to calculate the number of reliability verification tests N under a given c, and the (N, c) verification plan is formulated accordingly;

[0126] Step 3: Conduct reliability verification test and give verification conclusion. Conduct reliability verification test on the software according to the (N,c) verification plan formulated in step 2, repeat the specified task N times independently, and record the number of execution failures r. If r≤c, the software passes the reliability verification and is qualified; otherwise, the software is rejected.

[0127] Finally, the embodiment of the present application illustrates the proposed Bayesian reliability verification scheme through a real failure data set, and selects two typical Bayesian schemes to compare with the proposed Bayesian scheme to examine the effectiveness of the proposed scheme in reducing the number of test cases.

[0128] The example software is a large medical record system. During the development period, it has undergone multiple internal tests and version updates. Table 1 shows the test data recorded during a version update. A total of 17 weeks of reliability growth testing were carried out in a weekly manner to eliminate defects. A total of 204 software failures were recorded:

[0129] Table 1 Reliability growth test data of large medical record system (time unit: week)

[0130]

[0131] 1) Point Estimate of Software Success Rate

[0132] The growth test data provided in Table 1 is used as the prior information before software delivery acceptance. Formula (1) is used to fit the reliability growth process, and the maximum likelihood estimation method is used to estimate the parameters of the growth model, thereby calculating the software reliability The mean square error MSE and the correlation index R-square are used to measure the fitting effect of the model. The calculation results are shown in Figure 2 and Table 2, column 3:

[0133] Table 2 Point estimate of software success rate and validity test table of estimate

[0134]

[0135] Figure 2 is a comparison chart of the actual test data provided by the embodiment of the present application and the fitting data of the reliability growth model; combined with Table 2 and Figure 2 It can be seen that the growth model has a good fitting effect on the growth test data set, that is, the software success rate point estimate It has a high estimation accuracy, so it can ensure that the Bayesian solution based on this point estimate has good credibility.

[0136] 2) Develop a Bayesian validation scheme

[0137] Assuming that the given reliability index (R0, γ) is (0.95, 0.01), the estimated value of the hyperparameter (u0, v0) of the Beta distribution is calculated using formula (10) as (6.750 0, 0.533 2). The Bayesian verification scheme for software reliability is formulated according to formula (12), and the calculation results are shown in Table 3. In order to examine the effectiveness of the proposed Bayesian scheme in reducing the number of test cases, a Bayesian verification scheme based on prior moments and a Bayesian verification scheme without prior information are also given for comparison, and the calculation results are shown in Table 3:

[0138] Table 3 Comparison of Bayesian verification schemes for software reliability

[0139]

[0140] In Table 3, EX represents the expected value of the number of successfully executed test cases. It can be seen from Table 3 that the scheme proposed in this application requires the least number of test cases among the three Bayesian verification schemes. In particular, in the case of no-failure verification with c equal to 0, the number of test cases required by the Bayesian scheme proposed in this application is reduced by 26.8% compared with the Bayesian scheme based on prior moments and by 33% compared with the Bayesian verification scheme without prior information.

[0141] (3) Experimental results analysis

[0142] In this embodiment, the main difference between the three Bayesian verification schemes is that the methods for determining the prior distribution are different, which is essentially the different methods for processing the prior information. The Bayesian verification scheme without prior information is a classic verification scheme proposed by Bayesian and Laplace. This method assumes that the product to be inspected has no historical prior information, so the prior distribution of the reliability parameter is determined to be a uniform distribution.

[0143] Obviously, this method does not utilize historical test information. Regardless of whether the software product has prior information or not, and regardless of the test intensity before product delivery, the success rate estimation result given is the same. Therefore, the estimation deviation of the true level of software reliability is the largest, and the number of use cases required for the developed Bayesian verification scheme is also the largest; the Bayesian verification scheme based on prior moments is widely used in engineering. This method uses the average value of the product's historical reliability growth test data to estimate the software reliability before delivery. Although this prior processing method is simple, the processing of information growth trend is relatively rough, which will cause the estimation result to be lower than the true value of the software reliability level before delivery. Therefore, the number of use cases required for the developed Bayesian verification scheme will be less than the Bayesian verification scheme without prior information, but the method is still conservative; the Bayesian scheme proposed in this application uses the software reliability growth model to give a point estimate of the software success rate before delivery, and verifies the validity of the estimation result. The processing of prior information with a growth trend is more sufficient and reasonable, so the developed Bayesian scheme requires the least number of use cases.

[0144] Figure 3 The architecture diagram of a Bayesian reliability verification system for discrete software provided in an embodiment of the present application is as follows: Figure 3 As shown, including:

[0145] The growth test data set acquisition module 310 is used to acquire a discrete data set for performing reliability growth test on the discrete software; the discrete data set includes: the number of test cases executed by each group of tests and the number of successfully executed test cases; combining the discrete data set and the non-homogeneous Poisson process in the discrete time domain, determining the discrete software reliability growth model; and based on the discrete software reliability growth model, estimating the probability that the discrete software will successfully perform the preset specified task again after the reliability growth test;

[0146] A prior distribution determination module 320 is used to set the prior distribution of the reliability of the discrete software in the reliability verification test phase to be a beta distribution, and estimate the parameters of the beta distribution based on the probability of successfully executing the preset specified task again and the maximum entropy method; the reliability is the probability that the discrete software successfully executes the specified task in the reliability verification test phase;

[0147] The posterior distribution determination module 330 is used to determine the posterior distribution of the reliability based on the prior distribution after the estimated parameters and Bayes' theorem; determine the number of test cases required for reliability verification test of the discrete software under a preset tolerable number of failures according to the posterior distribution;

[0148] The reliability verification test module 340 is used to perform reliability verification test on the discrete software according to the number of test cases and the preset tolerable number of failures.

[0149] It should be understood that the above-mentioned system is used to execute the methods in the above-mentioned embodiments. The implementation principles and technical effects of the corresponding program modules in the system are similar to those described in the above-mentioned methods. The working process of the system can refer to the corresponding process in the above-mentioned method and will not be repeated here.

[0150] Based on the method in the above embodiment, the embodiment of the present application provides an electronic device, such as Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other via the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the method in the above embodiment.

[0151] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.

[0152] Based on the method in the above embodiment, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method in the above embodiment.

[0153] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method in the above embodiment.

[0154] It is understandable that the processor in the embodiment of the present application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor can be a microprocessor or any conventional processor.

[0155] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0156] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions may be transmitted from a website site, a computer, a server or a data center to another website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0157] It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.

[0158] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A Bayesian reliability verification method for discrete software, wherein the discrete software records data of task execution failures in discrete time; characterized in that: include: Acquiring a discrete data set for performing reliability growth testing on the discrete software; The discrete data set includes: the number of test cases executed by each group of tests and the number of successfully executed test cases; combining the discrete data set and the non-homogeneous Poisson process in the discrete time domain to determine the discrete software reliability growth model; and based on the discrete software reliability growth model, estimating the probability that the discrete software will successfully execute the preset specified task again after the reliability growth test; The prior distribution of the reliability of the discrete software in the reliability verification test phase is set to beta distribution, and the parameters of the beta distribution are estimated based on the probability of successfully executing the preset specified task again and the maximum entropy method; the reliability is the probability that the discrete software successfully executes the specified task in the reliability verification test phase; the prior distribution for: ; ;in, is a Beta distribution, is the beta function, is the hyperparameter of the prior distribution; R Represents the reliability of discrete software in the reliability verification stage; the hyperparameters of the Beta distribution are determined by the maximum entropy method using the following binary equations: in, is the information entropy function of the Beta distribution; It represents the probability that the discrete software can successfully perform the specified task again after the reliability growth test. l Indicates the number of test groups included in the reliability growth test, Indicates i The number of test cases executed by the group test; Solving the above binary equations, the estimated hyperparameter numerical solution is , then the prior distribution after estimating the parameters ; Determine the posterior distribution of the reliability based on the prior distribution after the estimated parameters and Bayes' theorem; determine the number of test cases required for reliability verification testing of the discrete software under a preset tolerable number of failures according to the posterior distribution; The discrete software is subjected to reliability verification testing according to the number of test cases and the preset tolerable number of failures.

2. The method according to claim 1, characterized in that Determine the discrete software reliability growth model, including: A counting function in a discrete time domain is used to represent the number of defects detected in the executed test cases in the reliability growth test; the number of detected defects is the difference between the number of executed test cases and the number of successfully executed test cases; In combination with the counting function in the discrete time domain, a non-homogeneous Poisson process in the discrete time domain is used to represent the defect detection function in the reliability growth test; the defect detection function is related to the cumulative defect function, and the cumulative defect function is the expected function of the counting function; A discrete software reliability growth model is determined in combination with the defect detection function; wherein the discrete software reliability growth model is a reliability function of discrete software; the reliability function is the probability that the discrete software will not detect defects after executing at least one test case after accumulating multiple test cases.

3. The method according to claim 2, characterized in that The reliability growth model includes two parameters: the total number of initial defects and the defect detection rate. The two parameters are estimated based on the discrete data set using the maximum likelihood estimation method; the reliability function and the cumulative defect function are both related to the two parameters.

4. The method according to claim 2 or 3, characterized in that: The defect detection function is: in, Indicates that the discrete software executes n Test cases detected k The probability of a defect; D ( n ) indicates cumulative execution n The cumulative defect function of test cases, for The expected function of is a counting function in the discrete time domain, which represents the number of defects detected in the n test cases that have been executed; exp[ ] represents the natural exponential function; represents the total number of initial defects, It represents the defect detection rate; is the reliability function of discrete software, which refers to the cumulative execution n After the test cases, the discrete software executes h The probability that a test case will no longer find a defect.

5. The method according to claim 3, characterized in that: The two parameters are estimated using the maximum likelihood estimation method, including: Wherein, the reliability growth test includes l Group testing, Indicates Group Test Execution test cases, there are Use cases were successfully executed and the number of defects detected was ; Indicates Group Test Execution test cases, there are The use case was successfully executed; Indicates l Group Test Execution test cases, there are Use cases are executed successfully, 1≤ i ≤ l The data set ; represents the total number of initial defects, Indicates the defect detection rate.

6. The method according to claim 1 or 2, characterized in that: The discrete software reliability growth model is: in, is the reliability function of discrete software; exp{} represents the natural exponential function; D ( n ) indicates cumulative execution n The cumulative defect function of test cases, D ( n+h ) indicates cumulative execution n+h The cumulative defect function of the test cases; the reliability growth test includes l Group test, i Group Test Execution Test cases, when n Take the sum of all test cases executed in the reliability growth test , 1≤ i ≤ l ; h When taking 1, It represents the probability that the discrete software can successfully execute the specified task again after the reliability growth test.

7. The method according to claim 1, characterized in that Posterior distribution for: ； in, N The number of test cases executed in the reliability verification phase, S for N The number of test cases that are successfully executed among the test cases; When the tolerable number of failures is set to When , the number of test cases is determined by the following inequality: in, for r The increasing function of r Represents discrete software execution in the reliability verification phase N The number of defects detected when testing a test case; The maximum risk that the user can bear; R 0 indicates the lower limit of reliability acceptable to users; The minimum value when the above inequality holds true N as the number of test cases.

8. A Bayesian reliability verification system for discrete software, wherein the discrete software records data of task execution failures in discrete time; characterized in that: include: A growth test data set acquisition module, used to acquire a discrete data set for performing reliability growth test on the discrete software; The discrete data set includes: the number of test cases executed by each group of tests and the number of successfully executed test cases; combining the discrete data set and the non-homogeneous Poisson process in the discrete time domain to determine the discrete software reliability growth model; and based on the discrete software reliability growth model, estimating the probability that the discrete software will successfully execute the preset specified task again after the reliability growth test; A prior distribution determination module is used to set the prior distribution of the reliability of the discrete software in the reliability verification test phase to beta distribution, and estimate the parameters of the beta distribution based on the probability of successfully executing the preset specified task again and the maximum entropy method; the reliability is the probability that the discrete software successfully executes the specified task in the reliability verification test phase; the prior distribution for: ; ;in, is a Beta distribution, is the beta function, is the hyperparameter of the prior distribution; R Represents the reliability of discrete software in the reliability verification stage; the hyperparameters of the Beta distribution are determined by the maximum entropy method using the following binary equations: in, is the information entropy function of the Beta distribution; It represents the probability that the discrete software can successfully perform the specified task again after the reliability growth test. l Indicates the number of test groups included in the reliability growth test, Indicates i The number of test cases executed by the group test; Solving the above binary equations, the estimated hyperparameter numerical solution is , then the prior distribution after estimating the parameters ; A posterior distribution determination module, used to determine the posterior distribution of the reliability based on the prior distribution after the estimated parameters and Bayes' theorem; determine the number of test cases required for reliability verification testing of the discrete software under a preset tolerable number of failures according to the posterior distribution; The reliability verification test module is used to perform reliability verification test on the discrete software according to the number of test cases and the preset tolerable number of failures.

9. An electronic device, characterized in that: include: at least one memory for storing a computer program; At least one processor is used to execute the program stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method according to any one of claims 1 to 7.

Images