Test case test result prediction method and device, electronic equipment and computer program product

Abstract

The invention discloses a test result prediction method and device of a test case, electronic equipment and a computer program product, and the method comprises the steps: obtaining and analyzing the information of a tested system of a high-availability test, and obtaining the key information of the tested system; a test case of the tested system is obtained and analyzed, and an analysis result of the test case of the tested system is obtained and comprises a to-be-predicted test result type of the test case; according to the key information of the tested system and the test case, a test result prediction model corresponding to the to-be-predicted test result type of the test case is utilized to predict a test result of the test case, and the test result prediction model is constructed based on historical test data. According to the method, the prediction model is constructed by analyzing the historical test data, utilizing data modeling and the like, the historical test asset value is fully played, the test result of the test case is predicted according to the key information of the tested system, the accuracy of the prediction result is improved, and a clear expected result reference is provided for testers.

Description

Technical Field

[0001] The present application relates to the technical field of high availability testing, and in particular to a test result prediction method, device, electronic device, and computer program product for a test case. Background Art

[0002] High availability is one of the factors that must be considered in the design of IT system architecture today. It is crucial for many important systems, so it is very important to conduct high availability testing on the system.

[0003] However, in the traditional high availability testing process, there are two problems. First, at present, the prediction of high availability use case test results is not combined with the actual situation of the system under test, and is usually relatively general. The accuracy and reliability of the prediction results are low, and it is difficult to serve as a criterion for judging whether the actual execution results of the use case have passed or not. The test conclusion depends largely on the experience of the tester; second, in the current high availability testing, the system under test will have many defects caused by incorrect or unreasonable component configuration. These defects are often caused by the same reason, but they can only be discovered after a round of use case execution, which wastes a lot of manpower and fails to give full play to the value of historical test assets. Summary of the invention

[0004] The embodiments of the present application provide a method, device, electronic device, and computer program product for predicting test results of a test case to improve the accuracy of the test result prediction of the test case.

[0005] The present application embodiment adopts the following technical solutions:

[0006] In a first aspect, an embodiment of the present application provides a test result prediction method for a test case, the test result prediction method for the test case comprising:

[0007] Obtain and parse the system information of the high availability test to obtain the key information of the system;

[0008] Obtaining and parsing a test case of the system under test to obtain a parsing result of the test case of the system under test, wherein the parsing result of the test case of the system under test includes a test result type to be predicted of the test case;

[0009] According to the key information of the system under test and the test cases of the system under test, the test results of the test cases of the system under test are predicted using a test result prediction model corresponding to the test result type to be predicted of the test case, and the test result prediction model is constructed based on historical test data.

[0010] Optionally, the obtaining and parsing of the tested system information of the high availability test to obtain key information of the tested system includes:

[0011] Acquire and organize the tested system information of the high availability test to generate a tested system information table, wherein the tested system information table includes at least one of a system deployment table, a system component table, and a tested transaction table;

[0012] The measured system information table is parsed to extract key information of the measured system.

[0013] Optionally, the test result prediction method of the test case further includes:

[0014] Acquire historical test data, wherein the historical test data includes historical test results of the test case;

[0015] Preprocessing the historical test data to obtain preprocessing results of the historical test data, wherein the preprocessing results include a data summary table corresponding to different types of test results and a component recommended configuration set table;

[0016] Determine the test result type to be predicted for each test case in the high availability test case library according to the preprocessing result of the historical test data, and generate a library of test result types to be predicted for the test cases;

[0017] Constructing a test result prediction model according to the preprocessing results of the historical test data;

[0018] The step of obtaining and parsing the test case of the system under test to obtain the parsing result of the test case of the system under test includes:

[0019] Based on the test result type library to be predicted for the test case, the test case of the system under test is parsed to obtain the test result type to be predicted for the test case.

[0020] Optionally, the preprocessing the historical test data to obtain a preprocessing result of the historical test data includes:

[0021] Obtain historical test reports and historical test defect tables and parse them respectively to obtain parsing results of historical test data;

[0022] Merging and classifying the parsed results of the historical test data to obtain historical test data corresponding to different test result types;

[0023] Obtaining and parsing the system deployment table and the system component table to obtain the parsing results of the system deployment table and the system component table;

[0024] According to the historical test data corresponding to different test result types and the parsing results of the system deployment table and the system component table, the system deployment table and the system component table are respectively associated with different test results to generate data summary tables corresponding to different types of test results.

[0025] Optionally, predicting the test result of the test case of the system under test by using a test result prediction model corresponding to the test result type to be predicted of the test case according to the key information of the system under test and the test case of the system under test includes:

[0026] If the test result type to be predicted of the test case includes a TPS impact result, then according to the key information of the system under test and the test case of the system under test, the TPS impact result prediction model is used to predict the TPS impact result of the test case of the system under test;

[0027] If the test result type to be predicted of the test case includes a restart time result, then based on the key information of the system under test and the test case of the system under test, a restart time result prediction model is used to predict the restart time result of the test case of the system under test;

[0028] If the test result type to be predicted of the test case includes a node switching result, a node switching result prediction model is used to predict the node switching result of the test case of the system under test according to the key information of the system under test and the test case of the system under test.

[0029] Optionally, the TPS impact result includes a TPS decrease range, and predicting the TPS impact result of the test case of the system under test by using a TPS impact result prediction model according to the key information of the system under test and the test case of the system under test includes:

[0030] Searching in a TPS decrease range knowledge base according to the key information of the system under test and the test cases of the system under test;

[0031] The TPS decrease range corresponding to the test case is determined according to the search results.

[0032] Optionally, the TPS impact result includes TPS jitter time, the key information of the system under test includes component deployment information of the system under test, and predicting the TPS impact result of the test case of the system under test by using the TPS impact result prediction model according to the key information of the system under test and the test case of the system under test includes:

[0033] Get the weight value of each configuration item of the system component;

[0034] The TPS jitter time of the test case of the system under test is determined according to the component deployment information of the system under test and the weight values ​​of each configuration item of the system component.

[0035] Optionally, the key information of the system under test includes server resource information of the system under test, and predicting the restart time result of the test case of the system under test by using the restart time result prediction model according to the key information of the system under test and the test case of the system under test includes:

[0036] Get the original sample data set of the restart time result prediction;

[0037] Extracting corresponding sample data from the original sample data set according to the server resource information of the system under test;

[0038] Perform probability calculation on the sample data extracted to obtain the probability distribution curve;

[0039] The restart time result of the test case of the system under test is determined according to the probability distribution curve.

[0040] Optionally, the key information of the system under test includes component deployment information of the system under test, and predicting the node switching result of the test case of the system under test by using a node switching result prediction model according to the key information of the system under test and the test case of the system under test includes:

[0041] According to the component deployment information of the system under test, querying in the component switching rule library;

[0042] In the case where the component switching rule base contains a switching rule corresponding to the component deployment information of the system under test, the node switching result of the test case is determined according to the switching rule corresponding to the component deployment information of the system under test.

[0043] In a second aspect, an embodiment of the present application further provides a test result prediction device for a test case, the test result prediction device for the test case comprising:

[0044] A first acquisition unit is used to acquire and parse the tested system information of the high availability test to obtain key information of the tested system;

[0045] A second acquisition unit is used to acquire and parse the test case of the system under test to obtain the parsing result of the test case of the system under test, wherein the parsing result of the test case of the system under test includes the test result type to be predicted of the test case;

[0046] A prediction unit is used to predict the test results of the test cases of the system under test according to the key information of the system under test and the test cases of the system under test, using a test result prediction model corresponding to the test result type to be predicted of the test case, wherein the test result prediction model is constructed based on historical test data.

[0047] In a third aspect, an embodiment of the present application further provides an electronic device, including:

[0048] Processor; and

[0049] A memory arranged to store computer executable instructions, wherein when the executable instructions are executed, the processor executes the test result prediction method of any of the test cases described above.

[0050] In a fourth aspect, an embodiment of the present application further provides a computer program product, comprising a computer program or instructions, which, when executed by a processor, implements the test result prediction method of any of the aforementioned test cases.

[0051] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects: the test result prediction method of the test case in the embodiment of the present application first obtains the tested system information of the high availability test and parses it to obtain the key information of the tested system; then obtains the test case of the tested system and parses it to obtain the parsing result of the test case of the tested system, and the parsing result of the test case of the tested system includes the test result type to be predicted of the test case; finally, according to the key information of the tested system and the test case of the tested system, the test result prediction model corresponding to the test result type to be predicted of the test case is used to predict the test result of the test case of the tested system, and the test result prediction model is constructed based on historical test data. The test result prediction method of the test case in the embodiment of the present application analyzes historical test data and constructs a prediction model by means of data modeling, which gives full play to the value of historical test assets, predicts the test results of the high availability test case based on key information such as the deployment configuration of the tested system, improves the accuracy of the prediction result, and provides a clear reference for expected results for testers. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0053] Figure 1 A flowchart of a test result prediction method for a test case in an embodiment of the present application;

[0054] Figure 2 This is a schematic diagram of a process flow of historical test data in an embodiment of the present application;

[0055] Figure 3 A schematic diagram of a weight value calculation process for each configuration item of a system component in an embodiment of the present application;

[0056] Figure 4A schematic diagram of a TPS jitter time prediction process in an embodiment of the present application;

[0057] Figure 5 A schematic diagram of a sample data set construction process in an embodiment of the present application;

[0058] Figure 6 A schematic diagram of a prediction process of a restart time result in an embodiment of the present application;

[0059] Figure 7 A schematic diagram of a node switching result prediction process in an embodiment of the present application;

[0060] Figure 8 A schematic diagram of a process for establishing a component switching rule library in an embodiment of the present application;

[0061] Fig. 9 This is a schematic diagram of a component node switching rule directory in an embodiment of the present application;

[0062] Fig.10 A schematic diagram of the overall process of predicting the test results of a test case in an embodiment of the present application;

[0063] Fig.11 A schematic diagram of the structure of a test result prediction device for a test case in an embodiment of the present application;

[0064] Fig.12 This is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0066] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0067] In order to facilitate the understanding of the embodiments of the present application, the main technical terms involved in the present application are first introduced:

[0068] 1) High availability: High availability refers to the ability of a system to perform its functions without interruption, indicating the degree of availability of the system. Through design, the time that the system cannot provide services normally can be reduced. When a part of the system fails for various reasons, the system can be ensured to be continuously available and the system service failure time can be minimized.

[0069] 2) High availability testing: High availability testing is a non-functional test. It verifies whether the system can provide services normally (or provide services in a downgraded manner) by simulating situations when the system has an abnormality, and verifies whether the system can recover to the operating state before the abnormality occurs after the abnormality is recovered.

[0070] 3) Natural Language Processing (NLP): A discipline that uses computer technology to analyze, understand and process natural language, using computers as a powerful tool for language research. It conducts quantitative research on language information with the support of computers and provides language descriptions that can be used by both humans and computers.

[0071] 4) Data normalization: refers to the normalization of data. In some cases, in order to accurately reflect the actual situation, the original data must be processed and normalized. Data normalization has a great impact on the similarity coefficient. The calculation results of the normalized data are quite different from those of the unnormalized data. This is because the similarity coefficient depends on the position of the coordinate origin. After normalization, the coordinate origin moves, which greatly changes the angle between the samples.

[0072] 5) Naive Bayes: This method is a classification method based on Bayes theorem and the conditional independence assumption of features. Its characteristic is that it assumes that the appearance of all features is independent of each other and does not affect each other, and each feature is equally important.

[0073] 6) Boxplot: Also known as box-and-whisker plot or box plot, it is a statistical chart used to show the dispersion of a set of data and can be used to detect data outliers.

[0074] 7) Range normalization: It is a normalization method that scales the original data according to the difference between the maximum and minimum values. Its meaning is to map the range of the original data to [0,1] so that all data can have the same importance, which is convenient for comparison and processing.

[0075] 8) Entropy weight method: It is an objective weighting method. Its basic idea is to determine the objective weight according to the variability of the indicator. The principle is that the smaller the variability of the indicator, the less information it reflects, and the corresponding weight should be lower.

[0076] 9) TPS decline: refers to the deviation between the stable TPS before the abnormal operation and the stable TPS after the abnormal operation in the high availability test.

[0077] 10) TPS jitter time: refers to the time required for TPS to start jittering and return to a stable state after an abnormal operation in a high availability test.

[0078] At present, the expected results of high-availability test cases usually only provide a general description, which cannot provide testers with a clear reference for expected results. During the test execution process, the testers' knowledge and experience are completely relied upon to judge the rationality of the results, which makes it difficult to ensure the uniformity and accuracy of the test conclusions. In addition, in the current high-availability testing process, there are many repeated defects caused by component configuration, and these problems usually need to be discovered after a round of case execution, failing to give full play to the value of historical test assets.

[0079] There are currently only some optimization methods for high availability testing processes in other scenarios in the prior art, such as the patent document "Cloud Platform High Availability Testing Environment Construction Method, Testing Method and Equipment (Application Number CN202210663562.9)", which proposes a method for building a cloud platform high availability testing environment that can complete related tests such as distributed storage high availability, management node high availability, image warehouse high availability and computing node high availability of multiple nodes through a small number of physical machines. However, in the methods and technologies related to high availability, there is no involvement in parsing and processing historical test products, building a reliable high availability prediction model based on historical data, and no intelligent prediction of the expected results of the high availability use case of the current system based on the deployment status of the system under test.

[0080] It can be seen that the expected results of current high availability test cases are usually just simple and general overviews, which rely entirely on manual judgment based on test experience during the actual execution process. This has the following shortcomings:

[0081] 1) Currently, high availability testing mainly uses chaos testing, which focuses on the automatic execution of tests. It only simply collects, organizes and displays test results, lacks relevant judgment criteria, and cannot provide testers with a reference for expected results.

[0082] 2) The prediction of non-functional test results is currently focused on the field of performance testing. There is little research on predictions related to high availability testing. There is a lack of processes and related technologies for pre-prediction of cases, and the value of historical test assets has not been fully utilized.

[0083] Based on this, the present application embodiment provides a test result prediction method for a test case, such as Figure 1 As shown, a flow chart of a test result prediction method for a test case in an embodiment of the present application is provided, and the test result prediction method for the test case at least includes the following steps S110 to S130:

[0084] Step S110, obtaining and analyzing the tested system information of the high availability test to obtain key information of the tested system.

[0085] When predicting the test results of the test case, the embodiment of the present application needs to first obtain the tested system information of the high availability test, which may specifically include system deployment, system component information, and transaction information of the tested system, etc. The tested system information is parsed to extract key information that affects the prediction of the test results as the basis for subsequent test result predictions.

[0086] Step S120, acquiring and parsing the test case of the system under test to obtain the parsing result of the test case of the system under test, wherein the parsing result of the test case of the system under test includes the test result type to be predicted of the test case.

[0087] When predicting the test results of test cases, it is also necessary to obtain the test cases of the system under test, parse the test cases of the system under test, and extract information such as the components and nodes involved in each test case and the type of test results to be predicted.

[0088] Step S130, predicting the test results of the test cases of the system under test according to the key information of the system under test and the test cases of the system under test using a test result prediction model corresponding to the test result type to be predicted of the test case, wherein the test result prediction model is constructed based on historical test data.

[0089] According to the test result type to be predicted for each test case, the extracted key information of the system under test is used as the input of the test result prediction model corresponding to the test result type to be predicted for the test case. For example, when testing the Redis master node restart case, the component type (Redis), deployment status, node status, transaction impact, configuration status, and machine-related status are input into the corresponding test result prediction model to obtain the prediction result of the prediction model. Finally, the prediction results of each test case can be integrated and filled into the expected result field of the test case table, and finally a complete high-availability test case is output.

[0090] The test result prediction method for the test case in the embodiment of the present application analyzes historical test data and constructs a prediction model using data modeling and other methods, thereby giving full play to the value of historical test assets, and predicts the test results of high-availability test cases based on key information such as the deployment configuration of the system under test, thereby improving the accuracy of the prediction results and providing testers with a clear reference for expected results.

[0091] In some embodiments of the present application, the obtaining of the tested system information of the high availability test and parsing to obtain the key information of the tested system includes: obtaining the tested system information of the high availability test and organizing it to generate a tested system information table, the tested system information table including at least one of a system deployment table, a system component table, and a tested transaction table; parsing the tested system information table to extract the key information of the tested system.

[0092] The embodiment of the present application can organize the tested system information according to the test requirements to form three documents to be processed, namely, the system deployment table, the system component table and the tested transaction table. Then, the system deployment table, the system component table and the tested transaction table are parsed respectively to extract the key information of the tested system.

[0093] 1) System deployment table analysis: various key factors are obtained by parsing the table header. Each record in the table contains virtual machine related information and corresponding physical machine related information (if the node type is a physical machine, only physical machine related information is available); 2) System component table analysis: components used by the tested system and corresponding configuration and deployment methods; 3) Tested transaction table analysis: the name, transaction type, and transaction link of the selected transaction, and the transactions affected by each component used by the tested system are obtained, such as Redis: transaction A, transaction B.

[0094] It should be noted that the analysis of the system deployment table and the system component table in the prediction phase is consistent with the key information extraction process of the two tables of the test system in the historical test data. For details, please refer to the explanation of the analysis process of historical test data later.

[0095] In some embodiments of the present application, the test result prediction method for the test case also includes: obtaining historical test data, the historical test data including historical test results of the test case; preprocessing the historical test data to obtain preprocessing results of the historical test data, the preprocessing results including a data summary table corresponding to different types of test results and a component recommended configuration set table; constructing a test result prediction model based on the preprocessing results of the historical test data; determining the test result type to be predicted for each test case in a high availability test case library based on the preprocessing results of the historical test data, and generating a library of test result types to be predicted for the test cases; obtaining the test case of the system under test and parsing it to obtain the parsed result of the test case of the system under test includes: parsing the test case of the system under test based on the library of test result types to be predicted for the test case, and obtaining the test result type to be predicted for the test case.

[0096] Before predicting the test results of the test cases, the embodiment of the present application also needs to go through the stages of historical test data collection, processing, and model building.

[0097] 1) Data source collection:

[0098] Collect and organize the historical high availability test execution records, test defects, corresponding system deployment tables and system component tables. Among them, the high availability test execution records may include test type, use case type (component type or service name), use case name (including operation type, such as Kill / Reboot, etc.), test content, test process, test transaction and test results (including TPS phenomenon, restart time, node switching situation); test defects include problem type, phenomenon (including use case description), expected results, cause of the problem, and solution. The system deployment table may include virtual machine information (machine name, component type, application type (such as application server / file server / database server, etc.), cloud platform type, virtual machine hardware configuration, operating system (such as Windows / Linux, etc.)), physical machine information (server model (brand + model, such as: Inspur NF5688M6), operating system, server firmware (such as BIOS / UEFI / BMC, etc.), hardware configuration (such as processor / memory / RAID card, etc.)), and the virtual machine and physical machine are associated through the model of the host server. The system component table may include component type, deployment method, actual configuration and other information. Then, the high availability test cases, deployment status, system component status and tested transaction status of the system to be tested are obtained and sorted as required, wherein the tested transaction table includes transaction name, transaction link and transaction type. 3) Complete high availability test case library.

[0099] 2) Data preprocessing:

[0100] Analyze and process historical test assets, use natural language understanding technology to extract key information and classify the data based on the Naive Bayes algorithm. Finally, four types of test result summary tables are obtained, namely, node switching result summary table, restart time result summary table, TPS decrease amplitude result summary table, TPS jitter time result summary table and component recommended configuration set table.

[0101] Combined with the complete high availability test case library and historical test results, determine the test result prediction type of each high availability test case. For example, the prediction type of the expected result of the Redis master node restart case should include the restart time, node switching, and TPS impact.

[0102] Analyze and process the use cases, deployment, components and transactions of the system under test, and extract key information. Finally, the result prediction type, system deployment, components involved and the mapping relationship between the transactions under test and components of each high availability test case of the system under test are obtained.

[0103] 3) Data storage: Stores the library tables generated in the data preprocessing stage, including the node switching result summary table, the restart time result summary table, the TPS drop amplitude result summary table, the TPS jitter time result summary table, the component recommended configuration set table and the information of the system under test, etc.

[0104] 4) Establishment of prediction model structure: Based on the processed historical data, prediction models corresponding to different test result types are trained respectively, which may include TPS result prediction model, restart time prediction model and node switching prediction model.

[0105] In some embodiments of the present application, the preprocessing of the historical test data to obtain the preprocessing results of the historical test data includes: obtaining a historical test report and a historical test defect table and parsing them respectively to obtain the parsing results of the historical test data; merging and classifying the parsing results of the historical test data to obtain historical test data corresponding to different test result types; obtaining a system deployment table and a system component table and parsing them to obtain the parsing results of the system deployment table and the system component table; according to the historical test data corresponding to different test result types and the parsing results of the system deployment table and the system component table, the system deployment table and the system component table are respectively associated with different test results to generate data summary tables corresponding to different types of test results.

[0106] like Figure 2 As shown, a schematic diagram of a process flow of historical test data in an embodiment of the present application is provided, which may specifically include the following steps:

[0107] 1) Parse historical test reports and test defect tables: Extract keywords from reports and defects through natural language understanding to determine the components, nodes, node status, configuration, operation type, and test results (including TPS drop, TPS jitter time, restart time, and node switching) involved in each execution record and defect.

[0108] 2) Merge test results and classify them: Merge the defects corresponding to the test cases. That is, one case may involve multiple test results. After merging, the naive Bayes algorithm is used to classify the test results according to the content into restart time results, node switching results, and TPS impact results. Here, there may be a test case record included in all three types of results.

[0109] 3) Parse the system deployment table and associate it with the test results: Parse the system deployment table to obtain machine information, and associate it with the test results according to the node name to obtain the restart time result of each node.

[0110] 4) Generate a restart time result summary table: including node (machine) name, virtual machine related information, physical machine related information, restart time, etc.

[0111] 5) Parse the system component table and associate it with the test results: Parse the system component table to obtain the component information involved in the system, and associate the deployment method in the component table with the node switching result and TPS drop in the test results according to the component type. Then, associate the configuration in the component table with the TPS jitter time result in the test results according to the component type.

[0112] 6) Generate a node switching result summary table: including component type, node name, node status, deployment method, related configuration, abnormal operation type, node switching result, etc.

[0113] 7) Generate a summary table of TPS decrease results: including component type, node name, deployment method, abnormal operation type, TPS decrease, etc.

[0114] 8) Generate a TPS jitter time result summary table: including component type, component configuration, abnormal operation type, TPS jitter time, etc.

[0115] 9) Analyze and process the TPS jitter time result summary table: Analyze and extract the configuration of each component and the corresponding TPS jitter time according to the component type.

[0116] 10) Component recommended configuration set table: Use the box plot algorithm to reduce the noise of the data set, and finally obtain the historical range value [Pj1, Pj2] of each component configuration and the corresponding TPS jitter time T0.

[0117] In some embodiments of the present application, predicting the test results of the test cases of the system under test according to the key information of the system under test and the test cases of the system under test using the test result prediction model corresponding to the test result type to be predicted of the test cases includes: if the test result type to be predicted of the test case includes a TPS impact result, then according to the key information of the system under test and the test cases of the system under test, the TPS impact result prediction model is used to predict the TPS impact result of the test case of the system under test; if the test result type to be predicted of the test case includes a restart time result, then according to the key information of the system under test and the test cases of the system under test, the restart time result prediction model is used to predict the restart time result of the test case of the system under test; if the test result type to be predicted of the test case includes a node switching result, then according to the key information of the system under test and the test cases of the system under test, the node switching result prediction model is used to predict the node switching result of the test case of the system under test.

[0118] The test result types of the embodiments of the present application mainly include TPS impact results, restart time results and node switching results. The TPS impact results further include TPS drop amplitude and TPS jitter time. Different test result prediction models can be constructed for different types of test results. In the prediction stage, the types of test results to be predicted for different test cases may be the same or different. For example, some test cases only need to predict the TPS drop amplitude and TPS jitter time, while some test cases need to predict the TPS impact results, restart time results and node switching results at the same time. According to the type of test result to be predicted for each test case, the corresponding prediction model is adopted to predict the test results to ensure the accuracy and completeness of the prediction results.

[0119] In some embodiments of the present application, the TPS impact result includes a TPS decrease amplitude, and predicting the TPS impact result of the test case of the system under test based on the key information of the system under test and the test case of the system under test using a TPS impact result prediction model includes: searching in a TPS decrease amplitude knowledge base based on the key information of the system under test and the test case of the system under test; and determining the TPS decrease amplitude corresponding to the test case based on the search result.

[0120] For the prediction of use case test results of tested systems such as databases and middleware, we can first parse the TPS decrease result summary table, and build a TPS decrease knowledge base for databases and middleware based on component characteristics, using component type, deployment method, and abnormal operation type as key values ​​and the corresponding TPS decrease as value.

[0121] In the prediction stage, the corresponding TPS decrease is retrieved from the knowledge base according to the component type, deployment method, and abnormal operation type of the test case. For example, when Redis (component type) is deployed in a multi-master and multi-slave cluster mode (deployment method): when a shard standby node service fails (abnormal operation type), TPS is not affected; when a shard master node fails (abnormal operation type), the standby node switches to the master node and TPS is not affected; when both the master and standby nodes of a shard fail (abnormal operation type), related transactions are unavailable, etc. For application services, multi-instance services in the same center are mostly load sharing services, and there should be flow control normally. The value of "abnormal instance number / total instance number" can be determined as the upper limit of the TPS decrease.

[0122] In some embodiments of the present application, the TPS impact result includes TPS jitter time, the key information of the system under test includes component deployment information of the system under test, and predicting the TPS impact result of the test case of the system under test using a TPS impact result prediction model based on the key information of the system under test and the test case of the system under test includes: obtaining weight values ​​of each configuration item of the system component; determining the TPS jitter time of the test case of the system under test based on the component deployment information of the system under test and the weight values ​​of each configuration item of the system component.

[0123] When predicting the TPS jitter time of a test case, you can first determine the weight values ​​of each configuration item of the system component, such as Figure 3 As shown, a schematic diagram of a weight value calculation process of each configuration item of a system component in an embodiment of the present application is provided, which specifically includes the following steps:

[0124] 1) Divide the TPS jitter time result summary table data into a training set and a validation set, classify the training set by component, and obtain the TPS jitter time result set of a single component.

[0125] 2) Complement and matrix the TPS jitter time result set of the component: Complement the configuration items without values ​​in the set with any value in the recommended value set [Pj1, Pj2], and matrix the padded TPS jitter time result set to obtain a matrix Z, where row i represents different data samples and column j represents different configuration items of the component.

[0126] 3) Determine whether each configuration value Zij in the matrix Z is ∈ [Pj1, Pj2]. If it is within the range, proceed to step 4); if it is not within the range, proceed to step 5).

[0127] 4) Calculate the degree of deviation: the degree of deviation between the configuration item value and the recommended configuration value d(Zij)=0.

[0128] 5) Determine whether each configuration value Zij in the matrix Z is less than Pj1. If so, proceed to step 6), if not, proceed to step 7).

[0129] 6) Calculate the degree of deviation: the degree of deviation between the configuration item value and the recommended configuration value d(Zij)=dev(Zij)=(Zij-Pj1) / Pj1.

[0130] 7) Determine whether each configuration value Zij in the matrix Z is greater than Pj1. If so, proceed to step 8).

[0131] 8) Calculate the degree of deviation: the degree of deviation between the configuration item value and the recommended configuration value d(Zij)=dev(Zij)=(Zij-Pj2) / Pj2.

[0132] 9) The deviation degree matrix d is obtained by summarizing steps 4), 6), and 8), where dij∈[-∞, +∞] represents different data samples and lists the deviation degrees between different configuration items and the recommended configuration values.

[0133] 10) Data normalization: The deviation degree matrix d is processed by the range method. For dij∈[-∞,0] data: Dij=(dij-min(dij)) / (max(dij)-min(dij))+0.0001; for dij∈[0,+∞) data: Dij=(max(dij)-dij) / (max(dij)-min(dij))+0.0001; after processing, the matrix D is obtained, Dij∈(0,1].

[0134] 11) Calculate information entropy: First calculate the relative weight of each configuration item (column j), the proportion of the i-th sample value under the j-th configuration item to the value of the configuration item: Calculate the information entropy of each configuration item: Where k = 1 / ln(n), Ej∈[0,1).

[0135] 12) Calculate the weight of each configuration item: The weight of the jth configuration item

[0136] 13) The configuration item weight value Wj obtained above is verified using a validation set. Based on the verification result, the weight value is adjusted and optimized, and finally the weight values ​​of each configuration item of the output component are summarized.

[0137] After determining the weight values ​​of each component configuration item, the TPS jitter time of the test case can be predicted based on the weight values ​​of each component configuration item and the component deployment information of the system under test. Figure 4 As shown, a schematic diagram of a TPS jitter time prediction process in an embodiment of the present application is provided, which specifically includes the following steps:

[0138] 1) Input the component table of the system under test and obtain the configuration item information of each component.

[0139] 2) Determine whether the configuration item value is within the recommended configuration value set. If so, proceed to step 3); if not, proceed to step 4).

[0140] 3) If the configuration item value is within the recommended configuration value set and the predicted TPS jitter time is less than or equal to T0 (under the default recommended configuration, the transaction TPS jitter time is T0), go to step 9).

[0141] 4) Sort the configuration items of the component by weight value, take the top three configuration items by weight value, and search the TPS jitter time result summary table one by one for the historical sample data of the configuration item values ​​that are the same or have the smallest difference.

[0142] 5) Calculate the degree of deviation between the three sets of historical configuration data retrieved and the recommended configuration set to obtain three sets of deviation degree data.

[0143] 6) The three groups of deviation degree data are weighted and calculated according to the weight value of each configuration item to obtain the comprehensive score of each group of data.

[0144] 7) Select the transaction TPS jitter time corresponding to the set of configuration data with the highest comprehensive score in the TPS jitter time result summary table as the predicted value T.

[0145] 8) Determine whether the TPS predicted jitter time is less than or equal to the expected value T0, if so, proceed to step 9), if not, proceed to step 10).

[0146] 9) Output the predicted jitter duration T of the transaction TPS.

[0147] 10) It indicates that the configuration of the tested system components is abnormal, and the corresponding test defects in the TPS jitter time result summary table are associated.

[0148] 11) Alarm, output the abnormal configuration information and related defect information of the tested system components, and output the predicted jitter time T and expected value T0 of the system transaction TPS under the current abnormal configuration.

[0149] In some embodiments of the present application, the key information of the system under test includes server resource information of the system under test, and predicting the restart time result of the test case of the system under test using a restart time result prediction model based on the key information of the system under test and the test case of the system under test includes: obtaining an original sample data set for restart time result prediction; extracting corresponding sample data from the original sample data set based on the server resource information of the system under test; performing probability calculation on the extracted sample data to obtain a probability distribution curve; and determining the restart time result of the test case of the system under test based on the probability distribution curve.

[0150] For the prediction of the restart time result, the embodiment of the present application can first use the incremental extraction method to regularly scan the data in the restart result summary table, and synchronize the incremental data to the virtual machine data set X0 and the physical machine data set W0, perform data noise reduction on the data of X0 and W0, and then associate and bind the two data sets through the physical device model to form a full amount of original sample data set S0. Figure 5 As shown, a schematic diagram of the construction process of a sample data set in an embodiment of the present application is provided, and the specific processing process is as follows:

[0151] 1) Regularly scan the restart time result summary table, and use incremental extraction to extract incremental virtual machine resource attribute data, including virtual machine model, cloud platform type, operating system, virtual machine hardware configuration, host server model, etc., and synchronize them to virtual machine data set X0.

[0152] 2) The synchronization operation of X0 is detected, and the noise reduction operation is automatically triggered after the synchronization is completed. The box plot algorithm is used to filter discrete data to perform data noise reduction on the data set X0, and the noise reduction result is written into the data set X1.

[0153] 3) Scan the restart time result summary table regularly, and extract incremental physical machine resource attribute data, including server model, operating system, hardware configuration, machine firmware, etc., by incremental extraction, and synchronize them to the virtual machine data set W0.

[0154] 4) The synchronization operation of W0 is detected, and the noise reduction operation is automatically triggered after the synchronization is completed. The box plot algorithm is used to filter discrete data to perform data noise reduction on the data set W0, and the noise reduction result is written into the data set W1.

[0155] 5) The host server model in the data set X1 is consistent with the server model in the data set W1. This is used as the associated node to establish a mapping relationship between X1 and W1, forming a predicted original sample data set S0, which includes fields such as physical machine model, virtual machine model, and restart time.

[0156] When a new project starts, the server resource information of the detailed deployment is obtained according to the deployment resource information collection table, and the structured data of the deployment information is obtained after parsing, and the virtual machine model and physical machine model of all services are determined. Sample data S1 and S2 of the corresponding machine model are extracted from S0, and the probability of the sample data X in the data set is calculated to obtain the distribution curve of its probability P. The expected value calculation formula is used To calculate the expected value of X as the prediction benchmark value, take the time t1 corresponding to the left boundary value of the probability interval and the time t2 corresponding to the right boundary value as the upper and lower limits of the prediction result. For example, the expected restart benchmark time is 39 seconds, and the fluctuation range is [20, 40] seconds.

[0157] like Figure 6 As shown, a schematic diagram of a prediction process of a restart time result in an embodiment of the present application is provided, which specifically includes the following steps:

[0158] 1) After obtaining and parsing the server resource information, the structured data of the system deployment information is obtained, and the machine category of each machine is determined.

[0159] 2) Determine the specific physical machine model based on the detailed deployment list information of the machine.

[0160] 3) Extract sample data corresponding to the machine model from S0 to form a temporary data set S1 of the physical machine model.

[0161] 4) Determine the specific virtual machine model based on the detailed deployment list information of the machine.

[0162] 5) Trace the host information and determine the specific physical machine model.

[0163] 6) Extract sample data corresponding to the machine model from S0 to form a temporary data set S2 for the physical machine model.

[0164] 7) With time as the horizontal axis, perform probability calculation on the sample data X in the data set to obtain the distribution curve of its probability P.

[0165] 8) Calculate the expected value of X E[X] as the prediction reference value through the expected value calculation formula, and take the time t1 corresponding to the left boundary value of the probability interval and the time t2 corresponding to the right boundary value as the upper and lower limits of the prediction result.

[0166] In some embodiments of the present application, the key information of the system under test includes component deployment information of the system under test, and predicting the node switching result of the test case of the system under test using a node switching result prediction model based on the key information of the system under test and the test case of the system under test includes: querying in a component switching rule library based on the component deployment information of the system under test; and determining the node switching result of the test case based on the switching rule corresponding to the component deployment information of the system under test when there is a switching rule corresponding to the component deployment information of the system under test in the component switching rule library.

[0167] like Figure 7 As shown, a schematic diagram of a node switching result prediction process in an embodiment of the present application is provided, which specifically includes the following steps:

[0168] 1) According to the input deployment of the system under test, query the component switching rules according to the corresponding node switching rule library directory.

[0169] 2) Determine whether there is a switching rule for the current deployment of the component. If so, go to 6); if not, go to 3).

[0170] 3) Output abnormal information to prompt manual judgment whether it is necessary to store the node switching rules of the component that conforms to the current deployment situation into the database. If necessary, turn to 4); if not, turn to 5).

[0171] 4) Expand the node switching rules of the current deployment of the component to the node switching rule library according to the corresponding directory and turn to 6).

[0172] 5) Prompt the user to modify the input deployment table of the system under test component, and then go to 1) after modification.

[0173] 6) Output the switching prediction results of different use cases of the component.

[0174] like Figure 8 As shown, a schematic diagram of the establishment process of a component switching rule library in an embodiment of the present application is provided, which specifically includes the following steps:

[0175] 1) Input the node switching summary result table output by historical data preprocessing, and normalize the table to obtain a component structured data set, including component deployment status - number of machines and master-slave status (see rule base directory), switching rule configuration parameters, abnormal operations, master-slave correspondence before abnormal steps, and master-slave correspondence after abnormal steps.

[0176] 2) Cluster the component structured data set. If the components, deployment conditions, abnormal operations, and the corresponding master-slave conditions before the abnormal steps are the same, but are not clustered into different groups, go to 5). If they are clustered into different groups, go to 3).

[0177] 3) Determine whether the node switching rule of the component in the current deployment involves the switching rule parameter configuration. If so, turn to 4). If not, output the abnormal information and end the process.

[0178] 4) After integrating the configuration parameters and node switching conditions into rules, turn to 5).

[0179] 5) The node switching rules of the component under the current deployment are added to the rule base under the corresponding directory according to the deployment situation.

[0180] like Fig. 9 As shown, a schematic diagram of a component node switching rule directory in an embodiment of the present application is provided. In addition to the root directory, the component node switching rule base directory has a total of four levels of directories. The first-level directory is divided into dual-machine deployment and cluster deployment (three or more) according to the number of component machine deployments. The second-level directory is divided according to the number of component master-slave deployments. The dual-machine deployment includes master-slave deployment and master-master deployment. The cluster deployment includes one master and multiple standby deployments, multiple masters and multiple standby deployments (two or more master nodes), and multiple master deployments. The third-level deployment is divided according to the component master-standby switching rules. The dual-machine master-standby deployment is divided into the standby not switching to the master and the standby can switch to the master. The one-master-multiple-standby and multi-master-multiple-standby modes of cluster deployment both include the standby not switching to the master, the standby switching to the master in sequence, and the standby switching to the master in election. Among them, the standby switching to the master in sequence of multiple masters and multiple standbys includes a deployment situation where one master node corresponds to one standby node.

[0181] The fourth-level directory is the node switching rules of the components under the corresponding deployment conditions. The third-level directory has the rules of switching from standby to master in order and the rules of standby election to master according to the component classification. At the same time, the rules of standby election to master take into account the brain split phenomenon (that is, before and after the node is restored, there can only be one master node in the corresponding master-slave relationship). The node switching rules under the other three-level directories are general rules, regardless of component type. The general rules for dual-machine / one master and one standby / standby does not switch to master, cluster deployment / one master and multiple standbys / standby does not switch to master, and cluster deployment / multiple masters and multiple standbys / standby does not switch to master are "master node abnormality, The standby node remains the standby node, and remains the master node after the node recovers; if the standby node is abnormal, the master node remains the master node, and remains the standby node after the node recovers. The general rules for dual-machine / one master and one standby / standby switching to master are "the master node is abnormal, the standby node switches to the master node, and becomes the standby node after the node recovers; the standby node is abnormal, the master node remains the master node, and remains the standby node after the node recovers." The general rules for cluster deployment / multi-master and dual-machine deployment / master-master are "if one of the master nodes is abnormal, the remaining master nodes remain the master nodes, and remain the master nodes after the nodes recover."

[0182] Taking the component RocketMQ as an example, there are four types of high-availability deployments provided by the official website, namely dual machines / active-active, cluster / multi-active, cluster / multi-active-active-active / active-active-active without switching to active, and cluster / multi-active-active-active / active-active-active switching in sequence. If the deployment of the Rocketmq system under test is one of the first three deployments, the general rule is returned. If the deployment of the Rocketmq system under test is cluster / multi-active-active-active / active-active-active switching in sequence, the Rocketmq rules are queried in the fourth-level directory of the current deployment. Taking the deployment of three active and three active as an example, the switching rule of Rocketmq returned is "If the primary node of any group is abnormal, the standby node of the same group switches to the primary node, and becomes the standby node after the node recovers. If the standby node of any group is abnormal, the primary node of the same group remains unchanged, and remains the standby node after the node recovers."

[0183] Of course, the specific method of constructing the component node switching rules can be flexibly configured by those skilled in the art according to actual needs, and is not specifically limited here.

[0184] In order to facilitate the understanding of the above embodiments, Fig.10 As shown, a schematic diagram of the overall process of predicting the test results of a test case in an embodiment of the present application is provided. It mainly includes the following steps:

[0185] S1: Input the system information under test: Arrange the system information under test as required to form three documents to be processed, namely the system deployment table, the system component table and the transaction table under test.

[0186] S2-S3: Data analysis and output

[0187] 1) System deployment table analysis: The key factors are obtained by parsing the table header. Each record in the table contains information related to the virtual machine and the corresponding physical machine.

[0188] 2) System component table analysis: the components used by the system under test and the corresponding configuration and deployment methods.

[0189] 3) Analysis of the transaction table under test: the name, transaction type, and transaction link of the selected transaction. And obtain the transactions affected by each component used by the system under test, such as Redis: transaction A, transaction B.

[0190] S4: Input and parse the test cases of the system under test: parse the high availability test cases of the system under test, and extract the components, nodes and high availability test result types to be predicted involved in each test case based on the test case result type library.

[0191] S5: Prediction of expected results of high availability test cases: According to the type of results to be predicted for each use case, the extracted system-related information is used as the input of each prediction model.

[0192] S6: Expected result output, use case alarm and defect association: The predicted results of each use case are integrated and filled into the expected result column of the use case table, and finally a complete high availability test case is output. At the same time, an alarm is issued for the use case whose predicted result of TPS jitter time does not meet the expectation, and historical defects are associated.

[0193] In summary, the key points of the test result prediction method of the test case of this application are mainly:

[0194] 1) Analyze historical data, build prediction models using data modeling, and predict the test results of high-availability use cases based on the deployment and configuration of the system under test;

[0195] 2) Establish prediction rules for high-availability use case test results and sort out the corresponding influencing factors. Analyze and process historical test products from three aspects: TPS impact results, restart time, and node switching. Establish four knowledge bases: TPS drop range result summary table, TPS jitter time result summary table, restart time result summary table, and node switching result summary table;

[0196] 3) Build a prediction model for high-availability use case test results. Use probability distribution graphs, entropy weight methods and other means to build a TPS impact result prediction model, a restart time prediction model and a node switching prediction model based on the historical result knowledge base;

[0197] 4) The entropy weight method is used to calculate the weight value of each configuration item of the component, and the TPS jitter time is predicted based on the weight value of each configuration item and the current configuration of the system. When the prediction result shows an abnormality, the component configuration alarm information is prompted and historical related defects and solutions are automatically recommended, so as to repair related configuration problems in advance before the test is executed and prevent high-frequency defects.

[0198] The test result prediction method of the test case of the present application has achieved at least the following technical effects:

[0199] 1) The prediction model of multiple test results can intelligently generate the expected TPS drop, TPS jitter time, node restart time and node switching results of each test case, reducing the requirements for the experience level of testers;

[0200] 2) By calculating the weight values ​​of component high availability-related configuration items, the TPS jitter time is predicted according to the current system configuration. When the prediction result is abnormal, historical related defects and solutions are automatically recommended, and related configuration problems are repaired in advance before the test is executed, thus giving full play to the value of historical test assets and achieving high-frequency defect prevention.

[0201] The present application also provides a test result prediction device 1100 for a test case. Fig.11 As shown, a schematic diagram of the structure of a test result prediction device for a test case in an embodiment of the present application is provided, wherein the test result prediction device 1100 for the test case includes: a first acquisition unit 1110, a second acquisition unit 1120 and a prediction unit 1130, wherein:

[0202] The first acquisition unit 1110 is used to acquire and parse the tested system information of the high availability test to obtain key information of the tested system;

[0203] A second acquisition unit 1120 is used to acquire and parse the test case of the system under test to obtain the parsing result of the test case of the system under test, wherein the parsing result of the test case of the system under test includes the test result type to be predicted of the test case;

[0204] The prediction unit 1130 is used to predict the test results of the test cases of the system under test according to the key information of the system under test and the test cases of the system under test, using the test result prediction model corresponding to the test result type to be predicted of the test case, and the test result prediction model is constructed based on historical test data.

[0205] In some embodiments of the present application, the first acquisition unit 1110 is specifically used to: obtain and organize the information of the system under test of the high availability test to generate a system under test information table, wherein the system under test information table includes at least one of a system deployment table, a system component table, and a transaction table under test; and parse the system under test information table to extract key information of the system under test.

[0206] In some embodiments of the present application, the test result prediction device 1100 for the test case also includes: a third acquisition unit, used to acquire historical test data, the historical test data including historical test results of the test case; a preprocessing unit, used to preprocess the historical test data to obtain preprocessing results of the historical test data, the preprocessing results including a data summary table corresponding to different types of test results and a component recommended configuration set table; a determination unit, used to determine the test result type to be predicted for each test case in the high availability test case library according to the preprocessing results of the historical test data, and generate a library of test result types to be predicted for the test cases; and construct a test result prediction model according to the preprocessing results of the historical test data; the second acquisition unit 1120 is specifically used to: parse the test case of the system under test based on the library of test result types to be predicted for the test case, and obtain the test result type to be predicted for the test case.

[0207] In some embodiments of the present application, the preprocessing unit is specifically used to: obtain historical test reports and historical test defect tables and parse them separately to obtain parsing results of historical test data; merge and classify the parsing results of the historical test data to obtain historical test data corresponding to different test result types; obtain system deployment tables and system component tables and parse them to obtain parsing results of the system deployment tables and system component tables; according to the historical test data corresponding to different test result types and the parsing results of the system deployment tables and system component tables, associate the system deployment tables and the system component tables with different test results respectively to generate data summary tables corresponding to different types of test results.

[0208] In some embodiments of the present application, the prediction unit 1130 is specifically used to: if the test result type to be predicted of the test case includes a TPS impact result, then based on the key information of the system under test and the test case of the system under test, use a TPS impact result prediction model to predict the TPS impact result of the test case of the system under test; if the test result type to be predicted of the test case includes a restart time result, then based on the key information of the system under test and the test case of the system under test, use a restart time result prediction model to predict the restart time result of the test case of the system under test; if the test result type to be predicted of the test case includes a node switching result, then based on the key information of the system under test and the test case of the system under test, use a node switching result prediction model to predict the node switching result of the test case of the system under test.

[0209] In some embodiments of the present application, the TPS impact result includes a TPS decrease range, and the prediction unit 1130 is specifically used to: search in a TPS decrease range knowledge base according to the key information of the system under test and the test cases of the system under test; and determine the TPS decrease range corresponding to the test case according to the search results.

[0210] In some embodiments of the present application, the TPS impact result includes TPS jitter time, the key information of the system under test includes component deployment information of the system under test, and the prediction unit 1130 is specifically used to: obtain weight values ​​of each configuration item of the system component; determine the TPS jitter time of the test case of the system under test according to the component deployment information of the system under test and the weight values ​​of each configuration item of the system component.

[0211] In some embodiments of the present application, the key information of the system under test includes server resource information of the system under test, and the prediction unit 1130 is specifically used to: obtain an original sample data set for predicting the restart time result; extract corresponding sample data from the original sample data set according to the server resource information of the system under test; perform probability calculation on the extracted sample data to obtain a probability distribution curve; and determine the restart time result of the test case of the system under test according to the probability distribution curve.

[0212] In some embodiments of the present application, the key information of the system under test includes component deployment information of the system under test, and the prediction unit 1130 is specifically used to: query in a component switching rule library according to the component deployment information of the system under test; if there is a switching rule corresponding to the component deployment information of the system under test in the component switching rule library, determine the node switching result of the test case according to the switching rule corresponding to the component deployment information of the system under test.

[0213] It can be understood that the test result prediction device for the above-mentioned test case can implement the various steps of the test result prediction method for the test case provided in the aforementioned embodiment, and the relevant explanations about the test result prediction method for the test case are applicable to the test result prediction device for the test case, which will not be repeated here.

[0214] Fig.12 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Fig.12 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0215] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.12 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0216] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0217] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a test result prediction device for the test case at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0218] Obtain and parse the system information of the high availability test to obtain the key information of the system;

[0219] Obtaining and parsing a test case of the system under test to obtain a parsing result of the test case of the system under test, wherein the parsing result of the test case of the system under test includes a test result type to be predicted of the test case;

[0220] According to the key information of the system under test and the test cases of the system under test, the test results of the test cases of the system under test are predicted using a test result prediction model corresponding to the test result type to be predicted of the test case, and the test result prediction model is constructed based on historical test data.

[0221] The above application Figure 1 The method performed by the test result prediction device of the test case disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0222] The present application also provides a computer program product, including a computer program or an instruction, which is implemented when the computer program or the instruction is executed by a processor. Figure 1 The test result prediction method of the test case in the embodiment shown is specifically used to perform:

[0223] Obtain and parse the system information of the high availability test to obtain the key information of the system;

[0224] Obtaining and parsing a test case of the system under test to obtain a parsing result of the test case of the system under test, wherein the parsing result of the test case of the system under test includes a test result type to be predicted of the test case;

[0225] According to the key information of the system under test and the test cases of the system under test, the test results of the test cases of the system under test are predicted using a test result prediction model corresponding to the test result type to be predicted of the test case, and the test result prediction model is constructed based on historical test data.

[0226] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0227] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0228] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0229] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1The steps for the functions specified in one or more boxes.

[0230] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0231] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0232] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0233] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0234] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0235] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A test result prediction method for a test case, characterized in that: The test result prediction method of the test case includes: Obtain and parse the system information of the high availability test to obtain the key information of the system; Obtaining and parsing a test case of the system under test to obtain a parsing result of the test case of the system under test, wherein the parsing result of the test case of the system under test includes a test result type to be predicted of the test case; According to the key information of the system under test and the test cases of the system under test, the test results of the test cases of the system under test are predicted using a test result prediction model corresponding to the test result type to be predicted of the test case, and the test result prediction model is constructed based on historical test data.

2. The test result prediction method of the test case according to claim 1, characterized in that: The key information of the system under test obtained by obtaining and parsing the information of the system under test for the high availability test includes: Acquire and organize the tested system information of the high availability test to generate a tested system information table, wherein the tested system information table includes at least one of a system deployment table, a system component table, and a tested transaction table; The measured system information table is parsed to extract key information of the measured system.

3. The test result prediction method of the test case according to claim 1, characterized in that: The test result prediction method of the test case also includes: Acquire historical test data, wherein the historical test data includes historical test results of the test case; Preprocessing the historical test data to obtain preprocessing results of the historical test data, wherein the preprocessing results include a data summary table corresponding to different types of test results and a component recommended configuration set table; Determine the test result type to be predicted for each test case in the high availability test case library according to the preprocessing result of the historical test data, and generate a library of test result types to be predicted for the test cases; Constructing a test result prediction model according to the preprocessing results of the historical test data; The step of obtaining and parsing the test case of the system under test to obtain the parsing result of the test case of the system under test includes: Based on the test result type library to be predicted for the test case, the test case of the system under test is parsed to obtain the test result type to be predicted for the test case.

4. The test result prediction method of the test case according to claim 2 is characterized in that: The preprocessing of the historical test data to obtain the preprocessing result of the historical test data includes: Obtain historical test reports and historical test defect tables and parse them respectively to obtain parsing results of historical test data; Merging and classifying the parsed results of the historical test data to obtain historical test data corresponding to different test result types; Obtaining and parsing the system deployment table and the system component table to obtain the parsing results of the system deployment table and the system component table; According to the historical test data corresponding to different test result types and the parsing results of the system deployment table and the system component table, the system deployment table and the system component table are respectively associated with different test results to generate data summary tables corresponding to different types of test results.

5. The test result prediction method of the test case according to claim 1, characterized in that: The step of predicting the test result of the test case of the system under test by using a test result prediction model corresponding to the test result type to be predicted of the test case according to the key information of the system under test and the test case of the system under test includes: If the test result type to be predicted of the test case includes a TPS impact result, then according to the key information of the system under test and the test case of the system under test, the TPS impact result prediction model is used to predict the TPS impact result of the test case of the system under test; If the test result type to be predicted of the test case includes a restart time result, then based on the key information of the system under test and the test case of the system under test, a restart time result prediction model is used to predict the restart time result of the test case of the system under test; If the test result type to be predicted of the test case includes a node switching result, a node switching result prediction model is used to predict the node switching result of the test case of the system under test according to the key information of the system under test and the test case of the system under test.

6. The test result prediction method of the test case according to claim 5, characterized in that: The TPS impact result includes a TPS decrease range. The TPS impact result prediction model is used to predict the TPS impact result of the test case of the system under test according to the key information of the system under test and the test case of the system under test, including: Searching in a TPS decrease range knowledge base according to the key information of the system under test and the test cases of the system under test; The TPS decrease range corresponding to the test case is determined according to the search results.

7. The test result prediction method of the test case according to claim 5, characterized in that: The TPS impact result includes TPS jitter time, the key information of the system under test includes component deployment information of the system under test, and the TPS impact result prediction model is used to predict the TPS impact result of the test case of the system under test according to the key information of the system under test and the test case of the system under test, including: Get the weight value of each configuration item of the system component; The TPS jitter time of the test case of the system under test is determined according to the component deployment information of the system under test and the weight values ​​of each configuration item of the system component.

8. The test result prediction method of the test case according to claim 5, characterized in that: The key information of the system under test includes server resource information of the system under test, and the restart time result prediction model is used to predict the restart time result of the test case of the system under test according to the key information of the system under test and the test case of the system under test, including: Get the original sample data set for restart time result prediction; Extracting corresponding sample data from the original sample data set according to the server resource information of the system under test; Perform probability calculation on the sample data extracted to obtain the probability distribution curve; The restart time result of the test case of the system under test is determined according to the probability distribution curve.

9. The test result prediction method of the test case according to claim 5, characterized in that: The key information of the system under test includes component deployment information of the system under test, and predicting the node switching result of the test case of the system under test by using the node switching result prediction model according to the key information of the system under test and the test case of the system under test includes: According to the component deployment information of the system under test, querying in the component switching rule library; In the case where the component switching rule base contains a switching rule corresponding to the component deployment information of the system under test, the node switching result of the test case is determined according to the switching rule corresponding to the component deployment information of the system under test.

10. A test result prediction device for a test case, characterized in that: The test result prediction device of the test case comprises: A first acquisition unit is used to acquire and parse the tested system information of the high availability test to obtain key information of the tested system; A second acquisition unit is used to acquire and parse the test case of the system under test to obtain the parsing result of the test case of the system under test, wherein the parsing result of the test case of the system under test includes the test result type to be predicted of the test case; A prediction unit is used to predict the test results of the test cases of the system under test according to the key information of the system under test and the test cases of the system under test, using a test result prediction model corresponding to the test result type to be predicted of the test case, wherein the test result prediction model is constructed based on historical test data.

11. An electronic device, comprising: processor; as well as A memory arranged to store computer executable instructions, wherein when the executable instructions are executed, the processor executes the test result prediction method of any one of the test cases of claims 1 to 9.

12. A computer program product, comprising a computer program or instructions, wherein when the computer program or instructions are executed by a processor, the test result prediction method of any one of the test cases of claims 1 to 9 is implemented.

Images