A simulation regeneration method for software reliability test failure data

Abstract

The application discloses a simulation regeneration method for software reliability test failure data, and relates to the technical field of software testing. The method comprises the following steps: extracting an operation profile from test cases of system testing and validation testing, determining a main path and occurrence probability of each operation; cleaning, time and quantity converting the failure data, and generating continuous base data by polynomial fitting; and generating simulation regeneration failure data meeting the characteristic requirements of a Poisson process by a normalized sampling method. The application makes full use of existing test data, solves the problems of insufficient data and low data quality caused by limited test time, and improves the accuracy and efficiency of software reliability evaluation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of software testing, and in particular, to a simulation regeneration method for software reliability test failure data. BACKGROUND

[0002] With the rapid development of information technology and the continuous expansion of software application fields, software quality has become an important factor affecting the overall performance of the system and the user experience. Among them, software reliability as an important indicator to measure software quality, its evaluation results are directly related to the stability and security of software in the actual running environment. In order to verify whether the software can be stable under the specified use conditions, it is necessary to conduct comprehensive reliability testing on the software, and to systematically analyze the failure data generated during the testing process, so as to provide a scientific basis for subsequent reliability prediction, risk assessment and quality improvement.

[0003] However, the existing software reliability testing method has obvious shortcomings, mainly in the number and quality of failure data. First of all, due to the limitation of project progress and resource investment, software reliability testing is usually implemented after system testing and confirmation testing, and at this time the software has been preliminarily verified and has high stability, and it is difficult to naturally generate a large number of failure data in a short time. This lack of data quantity directly affects the accurate capture and evaluation of potential defects of the reliability evaluation model for software. Secondly, the failure data collected in the actual testing, testing and use process often has problems such as missing data, missing items or non-standard records, resulting in uneven data quality. Most existing reliability evaluation models, such as Schneidewind model, Musa-Okumoto model, Duane model, etc., require that the input data must meet the statistical hypothesis that the cumulative failure number conforms to the Poisson process, and the data obtained by traditional testing often cannot meet this key requirement due to its incompleteness and local concentration.

[0004] In addition, part of the failure data generated in the traditional test is often affected by the limit value and boundary value effect, and its data distribution cannot truly reflect the failure trend of the software in the actual use process, further weakening the application value of the data in the reliability evaluation. Therefore, how to realize the simulation regeneration of failure data through the in-depth processing and mathematical processing of existing test data under the conditions of limited test time and resources has become a key problem to be solved in software reliability engineering. SUMMARY

[0005] The application aims to provide a method for simulating and regenerating failure data with sufficient quantity, qualified quality and meeting Poisson process distribution requirements by extracting operation profiles, data cleaning, time and quantity conversion, polynomial fitting and normalized sampling, so as to effectively make up for the defects of insufficient data and low data quality in traditional testing, provide reliable data support for software reliability evaluation, and promote the landing and popularization of software reliability engineering in practical application.

[0006] The technical solution of the application is to provide a software reliability testing failure data simulation regeneration method, which comprises the following steps:

[0007] S1, extracting operation profiles from the test cases of system testing and validation testing, including determining software usage mode, screening main path test cases, identifying operation initiation roles, constructing operation set and calculating operation occurrence probability;

[0008] S2, fitting the failure data of system testing and validation testing into base data, including removing non-main path failure data, rearranging data according to operation flow, implementing time conversion and quantity conversion processing, and generating base data fitting function through polynomial fitting

[0009] S3, normalizing sampling of base data, sample screening through suggestion function p(t) and acceptance threshold alpha, obtaining simulated and regenerated failure data meeting the requirements of software reliability model in quantity and quality.

[0010] In any of the above technical solutions, further, the operation profile extraction in step S1 specifically comprises:

[0011] S11, obtaining all test cases from system testing and validation testing, and extracting software usage mode according to software requirement specification;

[0012] S12, based on the usage mode extracted in S11, screening main path test cases from the test cases of system testing and validation testing, the screening method is to group the test cases according to operation sequence, then to remove duplicates, to merge the test cases with the same operation flow, parameters and execution results, to select the test case with the earliest successful result in each group as the first successful sample, and to collect all first successful samples to form main path test cases;

[0013] S13, determining the initiation roles of each operation in the actual running process of software according to software usage mode and main path test cases, i.e. identifying the users or modules that actively initiate each operation in the actual running process of software;

[0014] S14. According to the operation initiating role determined in S13, the main path test cases corresponding to each role are classified and summarized to form an operation set for each role;

[0015] S15: Count the number of times each operation in S14 appears in the main path test case, and calculate the occurrence probability P of each operation based on the ratio of the number of samples of each operation to the total number of main path samples. i , the calculation formula is as follows:

[0016]

[0017] Among them, n i is the total number of samples of the main path test cases corresponding to the operation, and N is the total number of samples of the main path test cases corresponding to all operations.

[0018] In any of the above technical solutions, further, the failure data fitting in step S2 specifically includes:

[0019] S21. Eliminate failure data generated by non-main path test cases from failure data of system testing and confirmation testing;

[0020] S22. Based on the operation set determined in S1, rearrange the failure data generated by the remaining main path test cases according to the actual execution order of each operation, so that the arrangement order of the failure data is consistent with the actual software operation process;

[0021] S23. Based on the time unit and predetermined conversion ratio required for reliability evaluation specified by the user, the failure occurrence time recorded in the rearranged failure data in S22 is subjected to time conversion processing to convert the original time data into time data that meets the evaluation requirements;

[0022] S24. Based on the probability of occurrence of each operation calculated in S15, the number of invalid data rearranged in S22 is converted. The calculation formula is as follows:

[0023]

[0024] Among them, N 折算前 and N 折算后 are the number of failure data before and after conversion respectively;

[0025] S25. Using the failure data adjusted by time and quantity in S8 and S9, a polynomial fitting method is used to fit the data to obtain a continuous basic data fitting curve, which is used as the basic data fitting function. It can reflect the cumulative number of failures at any time t.

[0026] In any of the technical solutions above, further, the normalized sampling in step S3 comprises:

[0027] S31, determining the sample number n required by the normalized sampling, and providing a suggestion function ρ(t) meeting the requirement of Poisson process characteristics, a calculation constant M, and a sampling acceptance threshold α determined according to an expert judgment method;

[0028] S32, randomly sampling from the suggestion function ρ(t) to obtain a sample point s, calculating a difference value β of the base data fitting function and the suggestion function ρ(t) at the sample point s

[0029] S33, judging whether the absolute value β of the calculated difference value is less than the acceptance threshold α, if the absolute value β of the difference value is less than the acceptance threshold α, accepting the sample point as valid sampling data, otherwise discarding the sample point and returning to step S32 for re-sampling;

[0030] S34, repeating the above steps until the number of the collected sample points meeting the requirement reaches the sample number n required by the normalized sampling.

[0031] In any of the technical solutions above, further, the calculation constant M needs to meet and the calculation constant M is better to be close to 1.

[0032] The present application has the following advantages:

[0033] The present application realizes the purpose of simulating and regenerating software failure data by fully utilizing and mathematically processing the test cases and failure data generated in the system test and validation test process, thereby solving the defects of insufficient number of failure data, low data quality and not meeting the requirements of software reliability evaluation model in traditional tests.

[0034] Specifically, the prior art mainly relies on long-time and intensive testing to obtain failure data, which often cannot meet the required data quantity and quality for reliability evaluation; and the present application utilizes the existing data in system testing and validation testing, realizes the simulation and regeneration of failure data by extracting operation profiles and mathematical processing, which not only shortens the testing period, but also generates a large number of required failure data in a limited testing time.

[0035] The application accurately reflects the user operation mode and operation probability in the actual running process of software by extracting operation profiles from test cases, so that the subsequently generated data can truly reproduce the failure conditions that may occur when the user uses the software, fundamentally improving the representativeness and reliability of the data, and the regenerated failure data can be closer to the actual running environment; and the prior art generally ignores the analysis of the operation mode, and fails to reflect the real user usage characteristics in the data generation process.

[0036] The conventional method fails to effectively eliminate the data that does not conform to the actual use conditions due to limit values, boundary values and the like when processing failure data, resulting in uneven data distribution and not meeting the assumption of Poisson process; the application obtains base data that can truly reflect the cumulative trend of failure by cleaning, rearranging, time and quantity conversion, and polynomial fitting of the failure data, and then ensures that the generated data meets the statistical model requirements through standardized sampling. BRIEF DESCRIPTION OF DRAWINGS

[0037] The above and additional aspects of the application will become apparent and easily understood from the following description of the embodiments, in conjunction with the accompanying drawings, in which:

[0038] Figure 1 It is an operation profile extraction flowchart of a simulation regeneration method of failure data of software reliability test according to an embodiment of the application;

[0039] Figure 2 It is a base data fitting flowchart of a simulation regeneration method of failure data of software reliability test according to an embodiment of the application. DETAILED DESCRIPTION

[0040] In order to more clearly understand the above-mentioned purposes, features and advantages of the application, the application will be further described in detail below in conjunction with the drawings and specific embodiments. It should be noted that the embodiments of the application and the features in the embodiments can be combined with each other without conflict.

[0041] In the following description, many specific details are set forth in order to provide a thorough understanding of the application, but the application can also be implemented in other ways different from those described herein, therefore, the scope of protection of the application is not limited by the specific embodiments disclosed below.

[0042] In software development, it is often necessary to test whether the software is stable and reliable, and the important basis for judging this is the error or failure data in the testing process. Ideally, enough failure data should be collected in the test, but in reality this is often not the case. Before formal reliability testing, the software has already undergone other tests, and the software is relatively stable, so it is difficult to naturally generate a large amount of error data within a limited time, and the error information collected may be missing or not standardized, and many statistical methods require that the error data meet certain mathematical rules (such as the Poisson process), otherwise it cannot be directly used to evaluate software reliability.

[0043] As shown in Figure 1 and Figure 2 , the embodiment provides a simulation regeneration method for software reliability test failure data, which supplements or regenerates sufficient and appropriate error data by using existing test data, so as to better evaluate software reliability, and the method comprises the following steps:

[0044] S1, extracting an operation profile from test cases of system testing and validation testing.

[0045] Software reliability testing is a black box test like general software function testing, but software reliability testing is a test according to the actual use of software by users, and the actual use of users is often concentrated on the main path of the software; and system testing and validation testing are tests according to software requirements or structure, which will carefully test valid domains, invalid domains and boundary values.

[0046] The operation profile is a technology for describing the use of software, and is also the key to distinguishing software reliability testing from general software function testing. The operation profile models the use of software, including an operation set and an operation occurrence probability. In practical applications, the identification and construction of the operation profile require a large amount of manpower and time, so the present application proposes a method for extracting the operation profile from test cases of system testing and validation testing, and the specific steps comprise:

[0047] S11, obtaining all test cases from system testing and validation testing, and extracting a use mode of the software according to a software requirement specification.

[0048] S12, based on the use mode extracted in S11, screening main path test cases from the test cases of system testing and validation testing, the main path test cases being the most effective operation flow in the normal use process of the software, and the screening method being grouping the test cases according to operation sequences, then performing a deduplication process, merging test cases having the same operation flow, parameters and execution results, selecting the test case with the earliest successful result in each group as the first successful sample, and collecting all the first successful samples to form the main path test cases.

[0049] S13, determining the operation initiation role of each operation in the actual running process of the software according to the software usage mode and the main path test case, that is, identifying the user or module that initiates each operation in the actual running process of the software.

[0050] S14, according to the operation initiation role determined in S13, classifying and summarizing the main path test cases corresponding to each role to form an operation set of each role.

[0051] S15, counting the number of times of each operation in the main path test case, and calculating the occurrence probability P of each operation according to the proportion of the sample number of each operation to the total number of main path samples. i , the calculation formula is as follows:

[0052]

[0053] Wherein, n i is the total number of samples of the main path test case corresponding to the operation, and N is the total number of samples of the main path test case corresponding to all operations.

[0054] S2, fitting the failure data of system testing and validation testing as base data.

[0055] The biggest difference between the failure data generated in the system testing and validation testing stage and the failure data generated in the reliability testing is that the system testing and validation testing is intensive testing of software in a short time, and the failure occurrence time will be relatively concentrated, which is difficult to accurately reflect the trend of the number of failures changing with time in the reliability testing process. Moreover, there are many failures caused by limit values and boundary values in the failure data of system testing and validation testing, which is not consistent with the characteristics of reliability testing "testing according to the actual use of software by users". Therefore, the application proposes a method of fitting the failure data of system testing and validation testing as base data, and the specific steps include:

[0056] S21, eliminating the failure data generated by non-main path test cases in the failure data of system testing and validation testing.

[0057] S22, based on the operation set determined in S1, re-arranging the remaining failure data generated by the main path test cases according to the actual execution order of each operation, so that the arrangement order of the failure data is consistent with the actual software operation process.

[0058] S23, because the failure occurrence time recorded in the failure data and the actual reliability test required time unit may be different, according to the user specified reliability evaluation required time unit and the predetermined conversion ratio, the failure occurrence time recorded in the rearranged failure data in S22 is time conversion processing, the original time data is converted into the time data that meets the evaluation requirements.

[0059] S24, according to the operation probability calculated in S15, the number of failure data rearranged in S22 is converted, and the calculation formula is as follows:

[0060]

[0061] Where, N 折算前 And N 折算后 The number of failure data before and after conversion respectively.

[0062] S25, using the failure data adjusted by time and quantity conversion in S8 and S9, the data is fitted by using polynomial fitting method, a continuous base data fitting curve is obtained, which is used as base data fitting function Can reflect the cumulative failure number to any time t.

[0063] S3, the base data is normalized sampling, and the failure data of simulation regeneration which meets the requirements of software reliability model in quantity and quality is obtained.

[0064] At present, many widely used reliability evaluation models such as Schneidewind model, Musa-Okumoto model, Duane model and the like require that the failure data must meet the assumption of "cumulative failure number meets Poisson process", and the base data fitting function Although it is continuous, it is often complex and does not meet the Poisson process. This leads to that the generated base data cannot be directly used as the input of the reliability evaluation model, and must be further normalized sampling to meet the assumption requirements of the model. Therefore, the normalization sampling method of the base data is proposed in the application, the base data fitting function Is normalized, and samples meeting the Poisson process are generated, and the specific steps are as follows:

[0065] S31, determine the sample number n required for normalization sampling, and provide a recommended function ρ(t) meeting the characteristics requirements of Poisson process, a calculation constant M and a sampling acceptance threshold α determined according to expert judgment method, wherein the calculation constant M needs to meet for any t, And the closer the calculation constant M is to 1, the better.

[0066] S32, randomly sampling from the proposal function ρ(t) to obtain a sample point s, and calculating a base data fitting function the absolute value of the difference between the proposal function ρ(t) and the sample point s

[0067] S33, judging whether the absolute value of the difference is less than an acceptance threshold α, if the absolute value of the difference is less than the acceptance threshold α, accepting the sample point as valid sampling data, otherwise discarding the sample point and returning to step S32 for re-sampling.

[0068] S34, repeating the above steps until the number of sample points collected reaches the required number n of samples for normalization sampling.

[0069] Through this step, we select a group of samples from the previously smoothed base data, which not only has sufficient quantity, but also has a statistical distribution that meets the Poisson process required in software reliability evaluation, so that it can be directly used for software reliability testing.

[0070] The method generally provides a method for "regenerating" software failure data using existing test data. It first summarizes how users use the software from test records, then organizes and mathematically fits the collected error data to form a curve representing the trend of error accumulation. Finally, through a standardized sampling method, error data that meets the requirements of a specific statistical distribution is selected from the curve, thereby generating a set of sufficient and high-quality data for evaluating whether the software is reliable.

[0071] This method not only solves the problem of insufficient error data due to test time limitations, but also ensures that the generated data meets the requirements of various reliability evaluation models, thereby greatly improving the efficiency and accuracy of software reliability testing.

[0072] In summary, the present application provides a simulation and regeneration method for software reliability test failure data, which comprises:

[0073] S1, extracting an operation profile from test cases of system testing and validation testing, including determining a software usage mode, screening main path test cases, identifying an operation initiation role, constructing an operation set and calculating the probability of each operation.

[0074] S2, fitting failure data of system testing and validation testing into base data, including removing non-main path failure data, rearranging data according to operation flow, implementing time conversion and quantity conversion processing, and generating a base data fitting function through polynomial fitting

[0075] S3, normalizing sampling of the basic data, screening the samples through the suggestion function p(t) and the acceptance threshold alpha, and obtaining failure data of simulation reproduction meeting the requirements of the software reliability model in quantity and quality.

[0076] The steps in the application can be adjusted in sequence, combined and deleted according to actual needs.

[0077] The units in the device of the application can be combined, divided and deleted according to actual needs.

[0078] Although the application has been disclosed with reference to the accompanying drawings, it is understood that these descriptions are merely exemplary and are not intended to limit the application of the application. The protection scope of the application is defined by the appended claims, and can include various modifications, modifications and equivalent solutions made to the application without departing from the protection scope and spirit of the application.

Claims

1. A simulation regeneration method for software reliability test failure data, characterized in that: The method comprises: S1. Extracting operation profiles from test cases in system testing and confirmation testing, including determining software usage patterns, screening main path test cases, identifying operation initiating roles, constructing operation sets, and calculating the probability of each operation occurring. S2. Fit the failure data of system test and confirmation test into base data, including eliminating non-main path failure data, rearranging data according to operation process, implementing time conversion and quantity conversion processing, and generating base data fitting function through polynomial fitting S3. Perform normalized sampling on the base data and perform sample screening using the recommended function ρ(t) and the acceptance threshold α to obtain simulated regenerated failure data whose quantity and quality meet the requirements of the software reliability model. The normalized sampling in step S3 includes: S31. Determine the number of samples n required for standardized sampling, and provide a suggested function ρ(t) that meets the requirements of the Poisson process characteristics, a calculation constant M, and a sampling acceptance threshold α determined by expert judgment; S32, randomly sample from the proposed function ρ(t), obtain the sample point s, and calculate the base data fitting function The absolute value of the difference between the proposed function ρ(t) at the sample point s S33, determining whether the calculated absolute value β of the difference is less than the acceptance threshold α. If the absolute value β of the difference is less than the acceptance threshold α, the sample point is accepted as valid sampling data; otherwise, the sample point is discarded and the process returns to step S32 for random sampling again. S34. Repeat the above steps until the number of sample points that meet the requirements reaches the number of samples n required for standardized sampling.

2. The simulation regeneration method for software reliability test failure data according to claim 1, characterized in that: Extracting the operation profile in step S1 specifically includes: S11. Obtain all test cases from system testing and confirmation testing, and extract the software usage pattern by referring to the software requirements specification; S12. Based on the usage patterns extracted in S11, main path test cases are screened from the test cases of the system test and confirmation test. The screening method is to group the test cases according to the operation sequence, then perform duplicate removal, merge test cases with exactly the same operation process, parameters, and execution results, select the test case that obtains the earliest successful result in each group as the first successful sample, and aggregate all the first successful samples to form the main path test cases; S13. Determine the initiating role of each operation during the actual operation of the software based on the software usage pattern and the main path test case, that is, identify the user or module that actively initiates each operation during the actual operation of the software; S14. According to the operation initiating role determined in S13, the main path test cases corresponding to each role are classified and summarized to form an operation set for each role; S15: Count the number of times each operation in S14 appears in the main path test case, and calculate the occurrence probability P of each operation based on the ratio of the number of samples of each operation to the total number of main path samples. i , the calculation formula is as follows: Among them, n i is the total number of samples of the main path test cases corresponding to the operation, and N is the total number of samples of the main path test cases corresponding to all operations.

3. The simulation regeneration method for software reliability test failure data according to claim 2, characterized in that: The failure data fitting in step S2 specifically includes: S21. Eliminate failure data generated by non-main path test cases from failure data of system testing and confirmation testing; S22. Based on the operation set determined in S1, rearrange the failure data generated by the remaining main path test cases according to the actual execution order of each operation, so that the arrangement order of the failure data is consistent with the actual software operation process; S23. Based on the time unit and predetermined conversion ratio required for reliability evaluation specified by the user, the failure occurrence time recorded in the rearranged failure data in S22 is subjected to time conversion processing to convert the original time data into time data that meets the evaluation requirements; S24. Based on the probability of occurrence of each operation calculated in S15, the number of invalid data rearranged in S22 is converted. The calculation formula is as follows: Among them, N 折算前 and N 折算后 are the number of failure data before and after conversion respectively; S25. Using the failure data adjusted by time and quantity in S23 and S24, a polynomial fitting method is used to fit the data to obtain a continuous basic data fitting curve, which is used as the basic data fitting function. It can reflect the cumulative number of failures at any time t.

4. The simulation regeneration method for software reliability test failure data according to claim 1, characterized in that: The calculation constant M needs to satisfy that for any t, And the closer the calculation constant M is to 1, the better.

Images