Reliability Evaluation Method, System, Device and Storage Medium of Software System

Abstract

The embodiments of the present application provide a method, a system, a device and a storage medium for evaluating the reliability of a software system. The method obtains test process information and test package information through an auxiliary monitoring module, statistically analyzes the test process information according to preset evaluation indicators to obtain test process parameters, fuses the test process parameters with the test package information to generate auxiliary evaluation parameters, obtains test result information from the test package information, and calculates the reliability confidence level of the software system according to the auxiliary evaluation parameters and the test result information according to the preset reliability evaluation rules, realizing the accurate evaluation of the reliability of the software system. Since the auxiliary evaluation parameters and the test result information are combined, the influence of the test process information on the reliability evaluation of the software system is comprehensively considered, greatly improving the accuracy and objectivity of the reliability evaluation of the software system compared with the traditional reliability evaluation based only on the test result information.

Description

Technical Field

[0001] This application relates to the technical field of software testing, and particularly relates to a method, system, device, and storage medium for evaluating the reliability of a software system. Background Art

[0002] The reliability of a software system refers to the ability of the software system to complete the specified functions under given conditions and within a specified time, which is used to test the system's ability to handle faults and is a core indicator for measuring the quality of a software system.

[0003] Currently, the method for evaluating the reliability of a software system in software testing is usually to evaluate based on the test results, that is, to perform multi-functional tests on the software, obtain the test results, and evaluate the reliability of the software system according to the test results. For example, if the test results meet the software test requirements, it is determined that the software system has high reliability; if not, it is determined that the software system has defects. This method of evaluating reliability based on test results, since this test method is only a qualitative measurement, cannot meet the accurate quantification of the reliability of the software system, reduces the accuracy of reliability evaluation, and is not conducive to the improvement of the quality of the software system.

[0004] Application Content

[0005] The embodiments of this application provide a method, system, device, and storage medium for evaluating the reliability of a software system to solve the technical problem of low accuracy in evaluating the reliability of a software system caused by evaluating reliability based on test results.

[0006] On the one hand, this application provides a method for evaluating the reliability of a software system, which is applied to a reliability evaluation system. The reliability evaluation system includes an auxiliary monitoring module for monitoring the test process of the software system. The method includes:

[0007] Obtain the test process information and test package information of the software system through the auxiliary monitoring module;

[0008] Statistically analyze the test process information according to preset evaluation indicators to obtain test process parameters;

[0009] Fusion-process the test process parameters and the test package information to generate auxiliary evaluation parameters;

[0010] Obtain test result information from the test package information;

[0011] Calculate the reliability confidence level of the software system according to the auxiliary evaluation parameters and the test result information according to preset reliability evaluation rules.

[0012] On the one hand, the present application provides a reliability evaluation system. The reliability evaluation system includes an auxiliary monitoring module for monitoring the testing process of the software system, including:

[0013] An acquisition module for acquiring, through the auxiliary monitoring module, the testing process information and test package information of the software system;

[0014] An analysis module for statistically analyzing the testing process information according to preset evaluation indicators to obtain testing process parameters;

[0015] A fusion module for performing fusion processing on the testing process parameters and the test package information to generate auxiliary evaluation parameters;

[0016] An extraction module for obtaining test result information from the test package information;

[0017] A determination module for calculating the reliability confidence level of the software system according to the auxiliary evaluation parameters and the test result information according to preset reliability evaluation rules.

[0018] On the one hand, the present application provides a computer device, including a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor executes the steps in the reliability evaluation method of the above software system.

[0019] On the one hand, the present application provides a computer-readable medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps in the reliability evaluation method of the above software system.

[0020] The embodiment of the present application provides a reliability evaluation method for a software system. Through an auxiliary monitoring module, the testing process information and test package information of the software system are acquired. The testing process information is statistically analyzed according to preset evaluation indicators to obtain testing process parameters. The testing process parameters and the test package information are subjected to fusion processing to generate auxiliary evaluation parameters. The test result information is obtained from the test package information. According to the auxiliary evaluation parameters and the test result information, the reliability confidence level of the software system is calculated according to preset reliability evaluation rules, realizing the accurate evaluation of the reliability of the software system. Since the auxiliary evaluation parameters and the test result information are combined, the influence of the testing process information on the reliability evaluation of the software system is comprehensively considered. Compared with the traditional reliability evaluation based only on the test result information, the accuracy and objectivity of the reliability evaluation of the software system are greatly improved. Description of the Drawings

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0022] Among them:

[0023] Figure 1 It is a schematic diagram of the application scenario of the reliability evaluation method for the software system in an embodiment;

[0024] Figure 2 It is a flowchart of the reliability evaluation method for the software system in an embodiment;

[0025] Figure 3 It is a structural block diagram of the reliability evaluation system in an embodiment;

[0026] Figure 4 It is a structural block diagram of a computer device in an embodiment. Detailed implementation manners

[0027] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0028] The reliability evaluation method for the software system provided by this application can be applied in an application environment such as Figure 1 In this application environment, the terminal device communicates with the server through the network. Among them, the terminal device can be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, and portable wearable devices. The server can be implemented by an independent server or a server cluster composed of multiple servers.

[0029] The system framework 100 may include a terminal device, a network, and a server. The network is used to provide a medium for the communication link between the terminal device and the server. The network can include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0030] The user can use the terminal device to interact with the server through the network to receive or send messages, etc.

[0031] The terminal devices 101, 102, and 103 can be various electronic devices with a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Eperts Group Audio Layer III), MP4 (Moving Picture Eperts Group Audio Layer IV) players, laptop computers, desktop computers, and so on.

[0032] The server 105 can be a server that provides various services, such as a background server that supports the pages displayed on the terminal devices 101, 102, and 103.

[0033] It should be noted that the reliability evaluation method of the software system provided by the embodiments of the present invention is executed by the server. Correspondingly, the reliability evaluation system is set in the server.

[0034] It should be understood that Figure 1 the numbers of the terminal devices, the network, and the server in

[0035] are merely illustrative. According to the implementation requirements, there can be any number of terminal devices, networks, and servers. The terminal devices in the embodiments of the present invention can specifically correspond to the application systems in actual production. Figure 2 As shown in

[0036] In one embodiment, a reliability evaluation method for a software system is provided. The reliability evaluation method of the software system is applied to a reliability evaluation system. The reliability evaluation system includes an auxiliary monitoring module for monitoring the test process of the software system. The reliability evaluation method of the software system specifically includes the following steps:

[0037] Step 201, obtain the test process information and test package information of the software system through the auxiliary monitoring module.

[0038] Among them, the test process information includes the operation information carried out during the test process and the information received by the business logic layer, which is used to reflect the real-time test information of the software system during the test process. For example, the severity of a fault at a certain time point, the CPU occupancy rate at a certain time, etc. Specifically, it can be obtained by acquiring the information monitored by the performance test tool in the auxiliary monitoring module. The test package information includes the test item conclusion and the information that users care about, such as the number of test cases, test results, etc. Specifically, it can be obtained by acquiring the information monitored by the automatic test tool in the auxiliary monitoring module.

[0039] It can be understood that in this embodiment, by enabling the auxiliary monitoring module to obtain the test process information and the test package information, the parameters for reliability assessment are ensured to be more objective and comprehensive, so as to facilitate further processing based on the test process information and the test package information subsequently.

[0040] Step 202: Statistically analyze the test process information according to the preset evaluation indicators to obtain the test process parameters.

[0041] Among them, the preset evaluation indicators refer to the information attribute items preset for evaluating the reliability degree of the software system related to the test process. For example, fault severity, failure data, the number of faults, etc. The statistical analysis refers to summarizing the test process information corresponding to each preset evaluation indicator at different time points and then analyzing to obtain the test process parameters.

[0042] Specifically, according to the preset evaluation indicators, extract the corresponding data from the test process information, summarize the corresponding data at each time point to obtain the summary data corresponding to each preset evaluation indicator, and perform aggregation analysis on each summary data, such as weighted summation, taking the maximum value, minimum value or average value, etc., so as to obtain the test process parameters, thus realizing the quantification of the test process parameters, which is beneficial to improving the accuracy of subsequent reliability assessment.

[0043] Step 203: Fuse the test process parameters with the test package information to generate auxiliary evaluation parameters.

[0044] Among them, the auxiliary evaluation parameters are parameters used to reflect the reliability degree of the software system. Specifically, perform correlation analysis on the test process parameters and the test package information to generate auxiliary evaluation parameters. More specifically, according to the data items included in the auxiliary evaluation parameters, classify the test process parameters and the system test parameters respectively to obtain each relevant data item, and perform correlation on each data item, such as performing summation or taking the median, to obtain the result value of each data item, and perform aggregation processing on the result values of each data item to generate the auxiliary evaluation parameters, realizing the quantitative calculation of the auxiliary evaluation parameters.

[0045] Step 204: Obtain the test result information from the test package information.

[0046] Among them, the test result information refers to the conclusion information of the software system test. For example, test cases, the number of test cases, the test success rate, the test time, etc. Specifically, obtain the test report from the test package information, and obtain the test result information according to the test report.

[0047] Step 205: Calculate the reliability confidence level of the software system according to the auxiliary evaluation parameters and the test result information according to the preset reliability evaluation rules.

[0048] Among them, the preset reliability evaluation rules refer to the calculation rules preset for quantifying the reliability degree of the software system. For example, the calculation rule can be a reliability evaluation function, or a calculation formula, or a machine learning model, such as a regression model.

[0049] Specifically, take the auxiliary evaluation parameters and the test result information as the independent variables of the preset reliability evaluation rules, and the dependent variable of the preset reliability evaluation rules is the reliability confidence level of the software system, obtaining the quantitative data reflecting the reliability degree of the software system, and realizing the accurate evaluation of the reliability of the software system. It can be understood that this embodiment combines the auxiliary evaluation parameters and the test result information, thus comprehensively considering the influence of the test process information on the reliability evaluation of the software system. Compared with the traditional reliability evaluation based only on the test result information, it greatly improves the accuracy and objectivity of the reliability evaluation of the software system.

[0050] The above-mentioned reliability evaluation method of the software system obtains the test process information and test package information of the software system through the auxiliary monitoring module, statistically analyzes the test process information according to the preset evaluation indicators to obtain the test process parameters, fuses and processes the test process parameters and the test package information to generate auxiliary evaluation parameters, obtains the test result information from the test package information, and calculates the reliability confidence level of the software system according to the auxiliary evaluation parameters and the test result information according to the preset reliability evaluation rules, realizing the accurate evaluation of the reliability of the software system. Since the auxiliary evaluation parameters and the test result information are combined, the influence of the test process information on the reliability evaluation of the software system is comprehensively considered. Compared with the traditional reliability evaluation based only on the test result information, it greatly improves the accuracy and objectivity of the reliability evaluation of the software system.

[0051] In one embodiment, the preset evaluation metrics include system operation performance, fault severity, and failure data, and each evaluation metric corresponds to a preset statistical analysis rule; the test process information is statistically analyzed according to the preset evaluation metrics to obtain test process parameters, including: respectively extracting the parameters corresponding to system operation performance, fault severity, and failure data and the test time point data, to obtain a first parameter, a second parameter, a third parameter, and their respective corresponding test time points; according to each preset evaluation metric, the first parameter, the second parameter, and the third parameter are respectively calculated and analyzed according to the corresponding preset statistical analysis rule to obtain a first process parameter, a second process parameter, and a third process parameter; the test process parameters include the first process parameter, the second process parameter, the third process parameter, and their respective corresponding test time points.

[0052] Among them, the system operation performance is an indicator used to reflect the level of software system operation performance. For example, the memory usage rate and the CPU occupancy rate. The fault severity is an indicator used to reflect the severity of the faults that occur during the software system testing process. The failure data refers to the test data corresponding to the determined faults during the test process, and the system operation performance, fault severity, and failure data all correspond to a preset statistical analysis rule. The preset statistical analysis rule among them refers to a calculation rule preset for quantifying the parameters corresponding to each evaluation metric. The preset statistical analysis rule can be a calculation formula, a quantification function, or a quantification model. For example, the calculation formula for the preset statistical analysis rule of the test process parameters corresponding to the fault severity can be

[0053] 100 > a > b > c > 0

[0054] Among them, L2 is the test process parameter corresponding to the fault severity at a certain time point.

[0055] The test process parameters refer to the parameters corresponding to each preset evaluation metric. Specifically, the parameters corresponding to system operation performance, fault severity, and failure data and the test time point data are respectively extracted from the test process information to obtain a first parameter, a second parameter, a third parameter, and their respective corresponding test time points; according to each preset evaluation metric, the first parameter, the second parameter, and the third parameter are respectively calculated and analyzed according to the corresponding preset statistical analysis rule to obtain a first process parameter, a second process parameter, and a third process parameter; the test process parameters include the first process parameter, the second process parameter, the third process parameter, and their respective corresponding test time points. The test time point refers to the acquisition moment of each test process parameter. Taking the above calculation formula as an example, L2 is the second process parameter, a, b, and c are the second parameters, and a, b, and c are all constants between (0, 100).

[0056] Understandably, since the preset evaluation metrics all reflect information related to the testing process, in this embodiment, by extracting the parameters of the corresponding preset evaluation metrics and calculating the testing process parameters for their respective parameters, including the first process parameter, the second process parameter, the third process parameter, and their respective corresponding testing time points, quantitative analysis of the testing process is achieved, and the testing time points corresponding to each testing process parameter are considered, enabling more refined expression and analysis of the testing process parameters.

[0057] In one embodiment, the auxiliary evaluation parameters include failure density and failure intensity, the test package information includes the number of faults found and the fault discovery rate. The testing process parameters and the test package information are fused and processed to generate auxiliary evaluation parameters, including: analyzing the second process parameter and the corresponding testing time point in the testing process parameters to obtain the first failure intensity and the first failure density; analyzing the first process parameter, the third process parameter, and their respective corresponding testing time points in the testing process parameters to obtain the second failure intensity and the second failure density; calculating the failure intensity based on the first failure intensity, the second failure intensity, the number of faults found, and the fault discovery rate; calculating the failure density based on the first failure density, the second failure density, the number of faults found, and the fault discovery rate.

[0058] Among them, the failure density refers to the probability of a failure occurring within a certain period, and the failure intensity refers to the severity level of a failure within a certain period. Specifically, by analyzing the testing time point corresponding to the second process parameter, determining the duration t of the second process parameter, and according to the cumulative quantity S2 of the second process parameter L2, the first failure density can be calculated using the following formula:

[0059] C1 = S2 * (t / T)

[0060] C1 is the first failure density, T is the time of one period, and T > t. The first failure intensity can be calculated using the following formula:

[0061] K1 = ∑L2 / S2

[0062] K1 is the first failure intensity, and L2 is the value L2 corresponding to the second process parameter. Then, by analyzing the first process parameter, the third process parameter, and their respective corresponding testing time points in the testing process parameters, the second failure intensity and the second failure density are obtained. Among them, the larger the value L1 corresponding to the first process parameter, the better the system performance; the larger the value L3 corresponding to the third process parameter, the more failure data. Therefore, the first process parameter has an inverse proportional relationship with the first failure intensity and the first failure density, and the third process parameter has a direct proportional relationship with the first failure intensity and the first failure density. Therefore, the second failure density can be calculated using the following formula:

[0063] C2 = L1 / L3 + α

[0064] C2 is the second failure density, α is an adjustment value. For example, α = 0.8. The second failure intensity can be calculated using the following formula:

[0065] K2 = L1 / L3 + σ

[0066] K2 is the second failure intensity, σ is an adjustment value. For example, σ = 0.9. Then, based on the first failure density, the second failure density, the number of detected failures, and the failure detection rate, the failure density can be calculated. The failure density can be calculated using the following formula:

[0067] C = C1 + C2 + max(F / S2, M)

[0068] C is the failure density, F represents the number of detected failures, M represents the failure detection rate, and max(F / S2, M) represents taking the maximum value between F / S2 and M;

[0069] Based on the first failure intensity, the second failure intensity, the number of detected failures, and the failure detection rate, the failure intensity can be calculated. The failure intensity can be calculated using the following formula:

[0070] K = K1 + K2 + max(F / S2, M) * λ

[0071] K is the failure intensity, and λ is an adjustment value. In this embodiment, by fusing the test process parameters and the test package information, the quantitative calculation of the failure density and the failure intensity is realized. Compared with the traditional qualitative analysis that only considers the failure intensity, the accuracy and integrity of the reliability assessment are greatly improved.

[0072] In one embodiment, the test package information further includes test data, multiple test cases, and multiple test results corresponding to the test cases; obtaining the test result information from the test package information includes: statistically analyzing each test result to generate the test result information.

[0073] Specifically, a mapping table between different test results and test confidence levels can be pre - constructed, and the corresponding confidence levels are determined respectively according to the multiple test results corresponding to each test case as the test result information.

[0074] In one embodiment, according to the auxiliary evaluation parameters and the test result information, calculating the reliability confidence level of the software system according to the preset reliability assessment rules includes: determining the first confidence level corresponding to the failure intensity; determining the second confidence level corresponding to the failure density; determining the third confidence level corresponding to the test result information; calculating the reliability confidence level of the software system according to the first confidence level, the second confidence level, and the third confidence level.

[0075] Among them, the first confidence level refers to the quantified value of the fault intensity, the second confidence level refers to the quantified value of the fault density, and the third confidence level refers to the quantified value of the test result information. Specifically, the first confidence level can be C in the above embodiments, the second confidence level can be K in the above embodiments, or both C and K can be corrected to obtain the corrected first confidence level and second confidence level. Then, the reliability confidence level of the software system is calculated according to the first confidence level, the second confidence level, and the third confidence level. More specifically, it can be the mean value or the weighted sum result of the first confidence level, the second confidence level, and the third confidence level as the reliability confidence level of the software system, or the reliability confidence level can be determined according to the comprehensive analysis of the first confidence level and the second confidence level and the third confidence level based on the comprehensive analysis result.

[0076] In one embodiment, calculating the reliability confidence level of the software system according to the first confidence level, the second confidence level, and the third confidence level includes: obtaining the weights corresponding to the first confidence level and the second confidence level; performing weighted calculation according to the first confidence level, the second confidence level, and the corresponding weights to obtain a comprehensive confidence level; calculating the absolute difference between the comprehensive confidence level and the third confidence level; when the absolute difference is greater than a preset threshold, determining the reliability confidence level according to the comprehensive confidence level.

[0077] Specifically, obtaining the weights corresponding to the first confidence level and the second confidence level, performing weighted calculation according to the first confidence level, the second confidence level, and the corresponding weights to obtain a comprehensive confidence level, calculating the absolute difference between the comprehensive confidence level and the third confidence level. When the absolute difference is greater than a preset threshold, it indicates that the gap between the comprehensive confidence level determined based on the auxiliary evaluation parameters after the fusion of the test process information and the test package information and the third confidence level corresponding to the test result information is relatively large, indicating that there is a certain deviation in the test result. Therefore, the third confidence level is not considered, and the reliability confidence level is determined according to the comprehensive confidence level, thus avoiding the influence of the deviated third confidence level, and being more simple and convenient, greatly improving the determination efficiency and accuracy of the reliability confidence level.

[0078] In one embodiment, before calculating the fault density according to the first fault density, the number of faults found, and the fault discovery rate, it further includes: performing a timing analysis on the test process information to obtain a test timing sequence; updating and calculating the first fault density according to the test timing sequence; calculating the fault density according to the updated first fault density, the number of faults found, and the fault discovery rate.

[0079] Among them, the timing analysis refers to the process of decomposing the test process information in chronological order to obtain the test timing sequence. Updating and calculating the first fault density according to the test timing sequence, and calculating the fault density according to the updated first fault density, the number of faults found, and the fault discovery rate greatly improves the accuracy of the quantification of the fault density and further improves the accuracy of the reliability confidence level.

[0080] In one embodiment, the system running performance includes at least one of CPU occupancy rate, usage frequency, and usage fluency.

[0081] Specifically, the system running performance includes at least one of CPU occupancy rate, usage frequency, and usage fluency. In this way, during the quantification of the system performance, quantification can be performed according to one or a combination of the CPU occupancy rate, usage frequency, and usage fluency, improving the accuracy of the first process parameter corresponding to the system running performance.

[0082] As Figure 3 shown, in one embodiment, a reliability evaluation system is proposed. The reliability evaluation system includes an auxiliary monitoring module for monitoring the testing process of the software system, including:

[0083] An acquisition module 301 for acquiring, through the auxiliary monitoring module, the testing process information and test package information of the software system;

[0084] An analysis module 302 for statistically analyzing the testing process information according to preset evaluation indicators to obtain testing process parameters;

[0085] A fusion module 303 for performing a fusion process on the testing process parameters and the test package information to generate auxiliary evaluation parameters;

[0086] An extraction module 304 for obtaining test result information from the test package information;

[0087] A determination module 305 for calculating the reliability confidence level of the software system according to the auxiliary evaluation parameters and the test result information according to a preset reliability evaluation rule.

[0088] In one embodiment, the analysis module includes:

[0089] An extraction unit for respectively extracting the parameters corresponding to the system running performance, fault severity, and failure data and the test time point data from the testing process information to obtain a first parameter, a second parameter, a third parameter, and their respective corresponding test time points;

[0090] A first analysis unit for respectively calculating and analyzing the first parameter, the second parameter, and the third parameter according to each preset evaluation indicator according to the corresponding preset statistical analysis rules to obtain a first process parameter, a second process parameter, and a third process parameter; the testing process parameters include the first process parameter, the second process parameter, the third process parameter, and their respective corresponding test time points.

[0091] In one embodiment, the fusion module includes:

[0092] A second analysis unit for analyzing the second process parameter and the corresponding test time point in the test process parameters to obtain a first failure intensity and a first failure density;

[0093] A third analysis unit for analyzing the first process parameter, the third process parameter and their respective corresponding test time points in the test process parameters to obtain a second failure intensity and a second failure density;

[0094] A first calculation unit for calculating the failure intensity according to the first failure intensity, the second failure intensity, the number of detected failures and the failure detection rate;

[0095] A second calculation unit for calculating the failure density according to the first failure density, the second failure density, the number of detected failures and the failure detection rate.

[0096] In one embodiment, the extraction module includes: a statistics unit for statistically analyzing each of the test results to generate test result information.

[0097] In one embodiment, the determination module includes:

[0098] A first determination unit for determining a first confidence level corresponding to the failure intensity;

[0099] A second determination unit for determining a second confidence level corresponding to the failure density;

[0100] A third determination unit for determining a third confidence level corresponding to the test result information;

[0101] A third calculation unit for calculating the reliability confidence level of the software system according to the first confidence level, the second confidence level and the third confidence level.

[0102] In one embodiment, the third calculation unit includes:

[0103] An acquisition subunit for acquiring weights corresponding to the first confidence level and the second confidence level;

[0104] A first calculation subunit for performing weighted calculation according to the first confidence level, the second confidence level and the corresponding weights to obtain a comprehensive confidence level;

[0105] A second calculation subunit for calculating the absolute difference between the comprehensive confidence level and the third confidence level;

[0106] A determination subunit for determining the reliability confidence level according to the comprehensive confidence level when the absolute difference is greater than a preset threshold.

[0107] In one embodiment, the reliability evaluation system further includes:

[0108] A timing analysis module, configured to perform timing analysis on the test process information to obtain a test timing;

[0109] An update module, configured to perform an update calculation on the first failure density according to the test timing;

[0110] A calculation module, configured to calculate the failure density according to the updated first failure density, the number of detected failures, and the failure detection rate.

[0111] Figure 4 The internal structure diagram of a computer device in one embodiment is shown. The computer device may specifically be a server, and the server includes but is not limited to high-performance computers and high-performance computer clusters. As Figure 4 shown, the computer device includes a processor, a memory, and a network interface connected through a system bus. Among them, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor can implement the reliability evaluation method of the software system. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor can execute the reliability evaluation method of the software system. Those skilled in the art can understand that Figure 4 the structure shown in is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0112] In one embodiment, the reliability evaluation method of the software system provided by the present application can be implemented in the form of a computer program, and the computer program can run on a computer device as Figure 4 shown. Each program template constituting the reliability evaluation system can be stored in the memory of the computer device. For example, an acquisition module 301, an analysis module 302, a fusion module 303, an extraction module 304, and a determination module 305.

[0113] A computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps in the reliability evaluation method of the above-mentioned software system are implemented.

[0114] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the reliability evaluation method of the above software system are implemented.

[0115] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided in this application can include non-volatile and / or volatile memories. Non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0116] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0117] The above-described embodiments merely represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A method for evaluating the reliability of a software system, characterized in that Applied to a reliability evaluation system, the reliability evaluation system includes an auxiliary monitoring module for monitoring the testing process of the software system, and the method includes: Obtain the testing process information and test package information of the software system through the auxiliary monitoring module; Statistically analyze the testing process information according to preset evaluation indicators to obtain testing process parameters. Among them, the preset evaluation indicators include system operation performance, fault severity, and failure data. Each evaluation indicator corresponds to a preset statistical analysis rule, including: respectively extract the parameters and test time point data corresponding to the system operation performance, fault severity, and failure data from the testing process information to obtain the first parameter, the second parameter, the third parameter, and their respective corresponding test time points; according to each preset evaluation indicator, calculate and analyze the first parameter, the second parameter, and the third parameter respectively according to the corresponding preset statistical analysis rule to obtain the first process parameter, the second process parameter, and the third process parameter; the testing process parameters include the first process parameter, the second process parameter, the third process parameter, and their respective corresponding test time points; Fuse the testing process parameters with the test package information to generate auxiliary evaluation parameters. The auxiliary evaluation parameters include fault density and fault intensity. The test package information includes the number of faults found and the fault discovery rate, including: analyze the second process parameter and the corresponding test time point in the testing process parameters to obtain the first fault intensity and the first fault density; analyze the first process parameter, the third process parameter, and their respective corresponding test time points in the testing process parameters to obtain the second fault intensity and the second fault density; calculate the fault intensity according to the first fault intensity, the second fault intensity, the number of faults found, and the fault discovery rate; calculate the fault density according to the first fault density, the second fault density, the number of faults found, and the fault discovery rate; Obtain the test result information from the test package information; Calculate the reliability confidence level of the software system according to the auxiliary evaluation parameters and the test result information according to the preset reliability evaluation rule.

2. The reliability evaluation method of the software system according to claim 1, wherein The test package information further includes test data, multiple test cases, and multiple test results corresponding to the test cases; The obtaining the test result information from the test package information includes: Statistically analyze each of the test results to generate test result information.

3. The reliability evaluation method of the software system according to claim 1, characterized in that The calculating the reliability confidence level of the software system according to the auxiliary evaluation parameters and the test result information according to the preset reliability evaluation rule includes: Determine the first confidence level corresponding to the fault intensity; Determine the second confidence level corresponding to the fault density; Determine the third confidence level corresponding to the test result information; Calculate the reliability confidence level of the software system according to the first confidence level, the second confidence level, and the third confidence level.

4. The reliability evaluation method of the software system according to claim 3, wherein The calculating the reliability confidence level of the software system according to the first confidence level, the second confidence level, and the third confidence level includes: Obtain the weights corresponding to the first confidence level and the second confidence level; Perform weighted calculation based on the first confidence level, the second confidence level, and the corresponding weights to obtain a comprehensive confidence level; Calculate the absolute difference between the comprehensive confidence level and the third confidence level; When the absolute difference is greater than a preset threshold, determine the reliability confidence level according to the comprehensive confidence level.

5. The reliability evaluation method of the software system according to claim 1, characterized in that, Before calculating the fault density based on the first fault density, the second fault density, the number of faults found, and the fault discovery rate, it further includes: Perform a timing analysis on the test process information to obtain a test timing; Perform an updated calculation on the first fault density according to the test timing; Calculate the fault density based on the updated first fault density, the number of faults found, and the fault discovery rate.

6. The reliability evaluation method of the software system according to claim 1, wherein The system operation performance includes at least one of CPU occupancy rate, usage frequency, and usage fluency.

7. A reliability evaluation system, characterized in that The reliability evaluation system includes an auxiliary monitoring module for monitoring the test process of the software system, including: An acquisition module for obtaining, through the auxiliary monitoring module, the test process information and test package information of the software system; An analysis module for statistically analyzing the test process information according to preset evaluation indicators to obtain test process parameters, where the preset evaluation indicators include system operation performance, fault severity, and failure data, and each evaluation indicator corresponds to a preset statistical analysis rule, including: respectively extracting the parameters and test time point data corresponding to the system operation performance, fault severity, and failure data from the test process information to obtain the first parameter, the second parameter, the third parameter, and their respective corresponding test time points; according to each preset evaluation indicator, respectively performing calculation and analysis on the first parameter, the second parameter, and the third parameter according to the corresponding preset statistical analysis rule to obtain the first process parameter, the second process parameter, and the third process parameter; the test process parameters include the first process parameter, the second process parameter, the third process parameter, and their respective corresponding test time points; A fusion module for fusing and processing the test process parameters and the test package information to generate auxiliary evaluation parameters, where the auxiliary evaluation parameters include fault density and fault intensity, and the test package information includes the number of faults found and the fault discovery rate, including: analyzing the second process parameter and the corresponding test time point in the test process parameters to obtain the first fault intensity and the first fault density; analyzing the first process parameter, the third process parameter, and their respective corresponding test time points in the test process parameters to obtain the second fault intensity and the second fault density; calculating the fault intensity based on the first fault intensity, the second fault intensity, the number of faults found, and the fault discovery rate; calculating the fault density based on the first fault density, the second fault density, the number of faults found, and the fault discovery rate; An extraction module for obtaining test result information from the test package information; A determination module, configured to calculate a reliability confidence level of the software system according to the auxiliary evaluation parameter and the test result information according to a preset reliability evaluation rule.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, the steps of the reliability evaluation method of the software system according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, the steps of the reliability evaluation method of the software system according to any one of claims 1 to 6 are implemented.

Images