Fault-tolerance testing method, device, electronic device and storage medium
By filtering test case templates and parameters based on the attribute information of the application to be launched, and performing fault-tolerant testing, the problem of low testing efficiency when there are too many clusters is solved, and a method of rapid online launch is realized.
Patent Information
- Application Number
- CN202010865406.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-08-25
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2040-08-25
AI Technical Summary
When there are many clusters and many service applications, the test task is large and the testing efficiency is low, which extends the online time of newly developed applications.
By determining whether it is a trunk application based on the attribute information of the application to be launched, filtering the corresponding test case templates and determining the test parameters, performing fault tolerance tests, and deciding whether to publish to the online cluster for further testing based on the test results.
It reduces the number of test applications, saves test time, reduces the workload of test tasks, improves test efficiency, and shortens the launch time of newly developed applications.
Smart Images

Figure CN113778842B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a method, device, electronic equipment and storage medium for fault-tolerant testing. Background Art
[0002] For various applications, they are usually developed based on new requirements in order to optimize the application. However, all newly developed applications need to be tested and can be put online after the test is completed. The current testing method is usually: after the application is developed based on the requirements analysis, the staff needs to formulate a test plan for the application, and then compile the test plan into a test case and test the application.
[0003] In the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art:
[0004] Since each newly developed application requires a test plan to be formulated and corresponding test cases to be written, when there are many clusters and service applications, the workload of the test task will be large, the test efficiency will be low, and the launch time of the newly developed application will be extended. Summary of the invention
[0005] In view of this, the embodiments of the present invention provide a method, device, system and storage medium for fault-tolerant testing, which can solve the problem that when there are many clusters and service applications, the workload of the test task is large, the test efficiency is low, and the online time of newly developed applications is prolonged.
[0006] To achieve the above objective, according to one aspect of an embodiment of the present invention, a method for fault-tolerance testing is provided.
[0007] A fault-tolerant testing method of an embodiment of the present invention includes: judging whether the application to be launched is a backbone application based on the attribute information of the application to be launched; if the application to be launched is a backbone application, screening a test case template corresponding to the type of the application to be launched according to the type of the application to be launched; determining test parameters corresponding to the application to be launched based on the test function of the application to be launched; performing a fault-tolerant test on the application to be launched based on the test case template and the test parameters, and obtaining a test result.
[0008] In one embodiment, the test parameters include input data and a test algorithm corresponding to the application to be launched;
[0009] The performing a fault tolerance test on the application to be launched based on the test case template and the test parameters, and obtaining a test result, includes:
[0010] Determine the test case corresponding to the application to be tested according to the test case template, the input data and the test algorithm, and obtain the expected result corresponding to the application to be launched;
[0011] Through the test case, a fault tolerance test is performed on the application to be launched, and an execution result is obtained;
[0012] The test result of the application to be launched is determined according to the execution result and the expected result.
[0013] In yet another embodiment, determining the test result of the application to be launched according to the execution result and the expected result includes:
[0014] Determine whether the similarity between the execution result and the expected result reaches a preset threshold;
[0015] If so, the test result is determined to be a passed test; if not, the test result is determined to be a failed test.
[0016] In yet another embodiment, after performing the fault tolerance test on the to-be-released application based on the test case template and the test parameters and obtaining the test result, the method further includes:
[0017] If the test result is that the test passes, the application to be launched is published to the online cluster, and a cluster test is performed on the application to be launched through the online cluster to obtain a cluster test result;
[0018] If the test result is that the test fails, a notification message is sent, and the notification message includes the test result.
[0019] In yet another embodiment, after performing cluster testing on the to-be-released application through the online cluster and obtaining a cluster testing result, the method further includes:
[0020] Determine whether the test data in the cluster test includes a point embedding mark;
[0021] If not, the test data is stored; if so, the test data is not stored.
[0022] In yet another embodiment, further comprising:
[0023] If the application to be launched is not a backbone application, publishing the application to be launched to an online cluster;
[0024] A cluster test is performed on the application to be put online through the online cluster to obtain a cluster test result.
[0025] In yet another embodiment, the attribute information includes the identification, priority, and whether it is a backbone function of the application to be launched.
[0026] To achieve the above objective, according to another aspect of an embodiment of the present invention, a device for fault-tolerance testing is provided.
[0027] A fault-tolerant testing device according to an embodiment of the present invention includes: a receiving unit, used to determine whether the application to be launched is a backbone application based on the attribute information of the application to be launched; a screening unit, used to screen a test case template corresponding to the type of the application to be launched if the application to be launched is a backbone application; a determination unit, used to determine a test parameter corresponding to the application to be launched based on the test function of the application to be launched; and a testing unit, used to perform a fault-tolerant test on the application to be launched based on the test case template and the test parameters, and obtain a test result.
[0028] In one embodiment, the test parameters include input data and a test algorithm corresponding to the application to be launched;
[0029] The testing unit is specifically used for:
[0030] Determine the test case corresponding to the application to be tested according to the test case template, the input data and the test algorithm, and obtain the expected result corresponding to the application to be launched;
[0031] Through the test case, a fault tolerance test is performed on the application to be launched, and an execution result is obtained;
[0032] The test result of the application to be launched is determined according to the execution result and the expected result.
[0033] In yet another embodiment, the testing unit is specifically used for:
[0034] Determine whether the similarity between the execution result and the expected result reaches a preset threshold;
[0035] If so, the test result is determined to be a passed test; if not, the test result is determined to be a failed test.
[0036] In yet another embodiment, the testing unit is further used for:
[0037] If the test result is that the test passes, the application to be launched is published to the online cluster, and a cluster test is performed on the application to be launched through the online cluster to obtain a cluster test result;
[0038] The device also includes:
[0039] A sending unit is used to send a notification message if the test result is that the test fails, and the notification message includes the test result.
[0040] In yet another embodiment, the apparatus further comprises:
[0041] A judging unit, used to judge whether the test data in the cluster test includes a buried point mark;
[0042] The storage unit is used for storing the test data if no; and not storing the test data if yes.
[0043] In yet another embodiment, the apparatus further comprises:
[0044] A publishing unit, configured to publish the application to be launched to an online cluster if the application to be launched is not a backbone application;
[0045] The testing unit is further configured to perform a cluster test on the application to be put online through the online cluster to obtain a cluster test result.
[0046] In yet another embodiment, the attribute information includes the identification, priority, and whether it is a backbone function of the application to be launched.
[0047] To achieve the above objective, according to another aspect of an embodiment of the present invention, an electronic device is provided.
[0048] An electronic device according to an embodiment of the present invention comprises: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the fault-tolerant testing method provided by the embodiment of the present invention.
[0049] To achieve the above objective, according to another aspect of an embodiment of the present invention, a computer readable medium is provided.
[0050] A computer-readable medium according to an embodiment of the present invention stores a computer program, and when the program is executed by a processor, the method for fault-tolerant testing provided by the embodiment of the present invention is implemented.
[0051] One embodiment of the above invention has the following advantages or beneficial effects: In the embodiment of the present invention, for an application to be launched, it can be determined whether the application to be launched is a backbone application based on its attribute information. If it is a backbone application, a test case template corresponding to its type is screened, and the test parameters corresponding to it are determined to perform a fault tolerance test on the application to be launched. In the embodiment of the present invention, for an application to be launched, a fault tolerance test is performed only when it is a backbone application, which can reduce the number of applications to be tested, and the test case template corresponding to its type can be screened for the application to be tested, without the need to rewrite the test case for it, thereby saving time for application testing. Even when there are many clusters and service applications, the workload of the test task can be reduced and the test efficiency can be improved.
[0052] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.
[0054] Figure 1 is a schematic diagram of a main process of a method for fault-tolerant testing according to an embodiment of the present invention;
[0055] Figure 2 is a schematic diagram of an attribute information table of an application to be launched according to an embodiment of the present invention;
[0056] Figure 3 is a schematic diagram of a test information table according to an embodiment of the present invention;
[0057] Figure 4 is a schematic diagram of a main process of a cluster testing method according to an embodiment of the present invention;
[0058] Figure 5 is a schematic diagram of another main process of the fault tolerance testing method according to an embodiment of the present invention;
[0059] Figure 6 is a schematic diagram of main units of a fault-tolerance test apparatus according to an embodiment of the present invention;
[0060] Figure 7 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;
[0061] Figure 8 It is a schematic diagram of the structure of a computer system suitable for implementing the embodiment of the present invention. DETAILED DESCRIPTION
[0062] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.
[0063] It should be pointed out that the embodiments of the present invention and the features in the embodiments may be combined with each other if there is no conflict.
[0064] An embodiment of the present invention provides a method for fault-tolerant testing, which can be used in a scenario of testing a developed application, for example, a scenario of testing an application after development in a microservice system. In an embodiment of the present invention, the specific content of the application to be launched can be determined according to the scenario, and it can be a minimum unit that can be developed, for example, a functional module of a system, or a part of an application. In an embodiment of the present invention, the testing of an application developed in a microservice system is used as an example for explanation.
[0065] The embodiment of the present invention provides a method for fault tolerance testing, which can be executed by a terminal or a server. Figure 1 As shown, the method includes the following steps.
[0066] S101: Based on the attribute information of the application to be launched, it is determined whether the application to be launched is a backbone application.
[0067] The attribute information of the application to be launched is predetermined. In the embodiment of the present invention, the attribute information of the application to be launched can be predetermined before this step, and the attribute information of the application to be launched can include the identification, priority, whether it is a backbone function, etc. of the application to be launched.
[0068] The backbone application indicates that the function implemented by the application is the backbone function of the system. In the embodiment of the present invention, the application can be divided into the backbone application and the non-backbone application based on whether the function implemented by the application is the backbone function. Therefore, in the embodiment of the present invention, some developed applications can be split and then tested separately, such as splitting into a minimum unit that can be developed, that is, the application to be launched, so as to achieve accurate division of whether the application to be launched is a backbone application.
[0069] It should be noted that in a microservice system, the iterative development speed of each version of the application is fast and the number is large. In order to facilitate application management, each application to be launched can be managed digitally, for example, in a table, in which the attribute information of each application to be launched is stored, and the attribute information of each application to be launched is divided and saved in the table as a row, that is, one row of data is the attribute information of one application to be launched. The priority of the attribute information of the application to be launched, whether it is a backbone function, etc. can also be determined by developers or testers. In this step, the attribute information of the application to be launched can be obtained from the stored table.
[0070] Specifically, the table storing the attribute information of the application to be launched can be as follows: Figure 2 As shown. Figure 2 As shown, the attribute information may include the functional module to which the application to be tested belongs, the name of the application to be launched, the overview of the application to be launched, the priority of the application to be launched, the technical research description of the application to be launched, the demand analysis and prototype design description of the application to be launched, the version number of the application to be launched, the owner (developer) corresponding to the application to be launched, whether the application to be launched implements the main function, and the expected launch time of the application to be launched.
[0071] In this step, after determining the attribute information of the application to be launched, it can be determined whether the application to be launched is a backbone application based on whether the attribute information is a backbone function. If the application to be launched implements a backbone function, it is a backbone application; if the application to be launched does not implement a backbone function, it is not a backbone application and is a non-backbone application.
[0072] S102: If the application to be launched is a backbone application, a test case template corresponding to the type of the application to be launched is selected according to the type of the application to be launched.
[0073] In the embodiment of the present invention, a variety of test case templates are pre-configured. After the application to be tested is determined to be a backbone application in step S101, the test case templates corresponding to the application can be screened out.
[0074] Since the optimization of the backbone applications in the system or the development of new functions usually does not change much, in the embodiment of the present invention, the test case template can be configured based on the functions included in the backbone applications, that is, the backbone applications are divided into types with different functions, and then the corresponding test case template is configured for each type. In this step, for the application to be launched, the corresponding test case template can be filtered out based on the type to which it belongs. For example, if the backbone functions of a system include ordering and customer service, then corresponding test case templates will be configured for the ordering type and customer service type of the online applications to be launched. The type of the application to be tested can be determined based on the functional module to which it belongs in the attribute information of the application to be tested.
[0075] It should be noted that, in the embodiment of the present invention, for each type, the corresponding test case template can be obtained by analyzing and abstracting the functions implemented by it. When the application of a certain function is tested for the first time, it is necessary to write a corresponding test case template for the application. In the embodiment of the present invention, a type of test case template can be a basic test case written for the function of that type, and the basic test case can implement the test of the basic functions of the function, so it can be used as a test case template.
[0076] S103: Based on the test function of the application to be launched, determine the test parameters corresponding to the application to be launched.
[0077] Among them, since the functions implemented by different applications to be launched may not be exactly the same, the personalized data of the applications to be launched also needs to be combined when testing, so in this step, the test parameters corresponding to the applications to be launched are determined based on the test functions of the applications to be launched. The test parameters may specifically include the input data and test algorithms corresponding to the applications to be launched.
[0078] Since different input data and test algorithms may produce different test results, and the test results are predicted before the test, the application to be launched can determine the input data and test algorithm, as well as the expected results corresponding to the input data and test algorithm during the test. The input data may include the form and content of the data input during the test.
[0079] Taking the customer service response function implemented by the application to be launched as an example, different question forms and different deformation algorithms may produce different results during testing, so it is necessary to first determine the question form and deformation algorithm used in the test, as well as the corresponding expected results. Among them, the question form is the input data corresponding to the application to be launched during testing, and the deformation algorithm is the test algorithm used by the application to be launched during testing. Specifically, the deformation algorithm can include machine learning methods CNN, decision tree analysis, naive Bayes, logistic regression, etc.
[0080] In the embodiment of the present invention, the input data and test algorithms used by each type of application to be launched during the test can be pre-configured, and then the test parameters corresponding to the application to be launched are determined based on the function of the application to be launched. In addition, in the embodiment of the present invention, when the input data and test algorithms configured for each type of application to be launched include multiple types, the specific input data and test algorithms can be determined based on the input instructions of the tester or developer.
[0081] S104: Based on the test case template and the test parameters, a fault tolerance test is performed on the application to be launched, and a test result is obtained.
[0082] After determining the test case template and test parameters corresponding to the application to be launched, the test case corresponding to the application to be launched can be determined, so that the application to be tested can be subjected to a fault tolerance test.
[0083] Specifically, the test parameters may include input data and test algorithms corresponding to the application to be launched. The input data and test algorithms are brought into the test case template, that is, the test case template, input data and test algorithm are combined to determine the test case corresponding to the application to be tested. At the same time, in the embodiment of the present invention, it is also necessary to obtain the expected results corresponding to the application to be launched, so as to judge the execution results later.
[0084] By running the test cases, you can implement fault-tolerance testing of the application to be launched, and get the execution results after the test cases are run. Then, based on the execution results and expected results, you can determine the test results of the application to be launched.
[0085] Specifically, according to the execution results and expected results, the test results of the application to be launched can be determined by: judging whether the similarity between the execution results and the expected results reaches a preset threshold; if so, determining the test result as a passed test; if not, determining the test result as a failed test.
[0086] In the embodiment of the present invention, a fault-tolerant test is performed on the application to be launched, so the execution result after the test case is executed may not be exactly the same as the expected result, so a similarity threshold between the execution result and the expected result can also be set in the embodiment of the present invention. After the execution result is obtained, the degree of similarity between the execution result and the expected result can be determined first, and then the magnitude relationship between the similarity and the similarity threshold can be compared. If the degree of similarity between the execution result and the expected result is greater than the similarity threshold, the test result of the application to be launched is determined to be a passed test; if the degree of similarity between the execution result and the expected result is not greater than the threshold, the test result of the application to be launched is determined to be a failed test. The threshold can be specifically set to 90%.
[0087] In the embodiment of the present invention, the similarity between the execution result and the expected result can be calculated by a similarity algorithm, or can be determined by a developer or tester inputting a similarity instruction.
[0088] It should be noted that, since there may be multiple input data and test algorithms determined for an application to be launched, multiple test cases may be determined after being combined with the test case template. In this case, the application to be launched needs to test each test case, and then determine the test result based on the average value of the test results of each test case. For example, if the proportion of test passes in the test results of each test case reaches a preset value, the test result can be determined as a test pass; if the proportion of test passes in the test results of each test case does not reach a preset value, the test result can be determined as a test failure.
[0089] In the embodiment of the present invention, the test related information of the application to be launched can be stored in a table. The test related information may include test parameters, expected results, similarity thresholds, etc. Specifically, taking the function of the application to be launched as customer service response as an example, the test related information is as follows: Figure 3 shown. Figure 3 In the code, application ID indicates the identification of the application to be launched; method indicates the input data of the application to be launched; deformation method indicates the test algorithm of the application to be launched; similarity threshold indicates the threshold of the similarity between the execution result and the expected result; status indicates the status of the test result of the application to be launched. If the test result of the application to be launched is a passed test, the status is valid; if the test result of the application to be launched is a failed test, the status is invalid; owner indicates the owner (developer) of the application to be launched. The test related information of different applications to be launched can be separated by blank lines, such as Figure 3 The row numbered 3 is empty and is used to distinguish different applications to be tested.
[0090] In an embodiment of the present invention, the application to be launched can set the test frequency during the test, so that the application to be tested can undergo multiple fault-tolerant tests, and then obtain a relatively stable overall test result. The application to be launched usually needs to be tested in three environments, namely: DEV, TEST and UAT (pre-release) environment. Among them, the DEV environment and the TEST environment are usually used by many people, and the positive test version is often released. The environment is unstable, so a lower test frequency can be set, such as once a day, while the version of the UAT environment is relatively stable, and a higher test frequency can be set, such as once every two hours. When the application to be launched performs multiple fault-tolerant tests, the overall test result is determined by the ratio of the number of test passes in each test result to the total number of test times. For example, if the ratio of the number of test passes in the test results of each test case to the total number of times reaches a preset value, the overall test result can be determined as a test pass; if the ratio of the number of test passes in the test results of each test case to the total number of times does not reach the preset value, the total test result can be determined as a test failure.
[0091] In an embodiment of the present invention, if the test result is that the test passes, the application to be launched can be released to the online cluster, and then the cluster test is performed in the online cluster to obtain the cluster test result; if the test result is that the test fails, a notification message can be sent to notify the developer or tester that the test fails so that he or she can re-develop it.
[0092] In the embodiment of the present invention, the applications to be launched are divided into backbone applications and non-backbone applications. Since the functions implemented by the non-backbone applications do not affect the main functions of the system, fault-tolerance testing is not required, and the applications are directly published to the online cluster for cluster testing. That is, only the process of steps S102, S103 and S104 in the embodiment of the present invention is executed on the backbone applications, thereby reducing the number of applications to be tested, and the test case templates corresponding to the types of the applications to be launched can be screened without rewriting the test cases for them, thereby saving time for application testing. Even when the number of clusters is large and the number of service applications is large, the workload of the test tasks can be reduced and the test efficiency can be improved.
[0093] It should be noted that after publishing the application to be launched to the online cluster, you can also perform tests within the cluster. The test frequency within the cluster can be set to execute regularly, and then determine whether the test within the cluster passes based on the test results.
[0094] Combine the following Figure 1 The embodiment shown in the figure illustrates the test process in the cluster after the application to be tested is released to the online cluster. Figure 4 As shown, the method includes the following steps.
[0095] S401: Perform cluster testing on the application to be launched in the online cluster.
[0096] Among them, the applications to be released to the online cluster include backbone applications and non-backbone applications. Cluster testing is required for both backbone applications and non-backbone applications in the online cluster.
[0097] In an embodiment of the present invention, a cluster test frequency can be set for backbone applications and non-backbone applications, and cluster testing can be performed according to the set cluster test frequency. At the same time, when the application to be launched is a backbone application, a heartbeat test can also be set. The heartbeat test refers to a test with a higher test frequency.
[0098] Specifically, when a high-frequency heartbeat test can be used for the backbone application, the test frequency can be set to be executed once every two minutes. The cluster test frequency can be set according to demand, for example, it can be set to be executed once every two hours. In the embodiment of the present invention, the backbone application can be cluster tested according to the cluster test frequency, and the heartbeat test can be performed according to the heartbeat test frequency, so as to strengthen the test of the backbone application, and when a problem occurs in the cluster test, the heartbeat test can enable the backbone application to still execute the test process, so as to ensure that the backbone application can stably realize the corresponding functions after going online.
[0099] In the embodiment of the present invention, the test cases used in the cluster test may be selected from a test case set, and the test case set may be pre-established or may be a used test case, such as the test case used in the fault tolerance test in step S104.
[0100] S402: Determine the overall cluster test result.
[0101] The cluster test result includes the test result of the cluster test and the test result of the heartbeat test. In the embodiment of the present invention, the overall cluster test result can be determined as passed only when the test result of the cluster test and the test result of the heartbeat test are both passed.
[0102] Since the functions of non-backbone applications do not affect the backbone functions, the cluster test can be a fault-tolerant test. That is to say, in the embodiment of the present invention, a threshold value of the cluster test can be set. For all cluster tests executed regularly, if the proportion of the number of test passes to the total number of test times reaches the threshold value, the test result of the cluster test can be determined. The threshold value can be set according to the requirements, for example, it can be set to 90%. The heartbeat test is a test of the backbone application, and usually the test result needs to be all passed to meet the requirements, that is, the heartbeat test executed regularly requires that the result of each test is a test pass before the test result of the heartbeat test can be determined as a test pass. Therefore, the test result of the cluster test is a test pass.
[0103] In the embodiment of the present invention, when the overall cluster test result is a test pass, it means that the application in the cluster can be launched, and the pass result can be notified to relevant personnel, such as developers, to execute the launch process. When the overall cluster test result is a test failure, it means that the application in the cluster needs to be further developed, and the failure result can be notified to the developers, who need to redevelop.
[0104] It should be noted that after the application to be launched is released to the online cluster, in order not to affect the user's use, it is necessary to perform diversion processing, such as controlling the user's entry into the application to be launched through the pin or venderID field in the chatinfo in the protocol to ensure that no user uses the application to be launched, and then enable users to use it through settings after the application to be launched passes the test.
[0105] S403: Save the test data.
[0106] In an embodiment of the present invention, the test data after the test will be stored, and the test data may include the overall cluster test results and the relevant data of each test execution. Since there are many tests, in order to reduce the generation of test garbage data, fixed-numbered test cases and test accounts can be used during the test process. For the test results, a report can be generated to notify relevant personnel. In order to reduce garbage data and reduce the impact of test data on report generation, each test case can be marked with a buried point in an embodiment of the present invention. The test data corresponding to the test case with the buried point mark may not be stored. Therefore, when storing test data, the test data for storage can be distinguished according to the buried point mark. Specifically, when storing the test data, it can be determined whether the test data includes a buried point mark. If so, it will not be stored; if not, the test data will be stored. Since the test data corresponding to the test case with the buried point mark may not be stored, it will not be used to generate reports and will not affect report generation.
[0107] It should be noted that punctuation marking can be implemented by setting a whitelist.
[0108] A test report can be generated after each test is executed. The test report can include information about the test cases of the application to be launched, execution time, test results, etc. The test report can be sent to relevant personnel by email or other means.
[0109] In the embodiment of the present invention, a cluster test is performed on the application to be launched in the online cluster to ensure that the function of the application to be launched can be stably realized after it is launched.
[0110] The testing of the application to be launched usually includes the testing performed by the developer and the testing performed by the tester. After the developer develops the application to be launched, the developer can perform the testing process by himself, and then the tester can perform the testing. The test scenario used in the embodiment of the present invention can be testing by the tester, that is, the developer's self-testing process is performed on the backbone application and the non-backbone application after the development is completed to ensure that the application to be launched can run. Then the backbone application tester passes Figure 1The method of the embodiment shown in the figure performs the test and publishes it to the online cluster after the test passes, instead of directly publishing the backbone application to the online cluster. In addition, for the backbone application and the non-backbone application, the test performed by the developer can be a fault-tolerant test. The specific test method can be as follows: Figure 1 In the embodiment shown, the test cases are determined and the fault tolerance test is performed. The specific overall test execution process of the application to be launched can be as follows: Figure 5 shown.
[0111] like Figure 5 As shown, in an embodiment of the present invention, the online application is distinguished into a backbone application and a non-backbone application. After analyzing and developing the application, it will be determined whether it is a backbone application or a non-backbone application, and then it will be released to the pre-release cluster after analysis and development. The pre-release cluster includes backbone applications and non-backbone applications. For the backbone application, after development, it is also necessary to determine the test case, wherein, if the corresponding function of the backbone application is the first time to perform the test, it is necessary to write a test plan and a test case; if the corresponding function of the backbone application is not the first time to perform the test, the test case can be determined according to the method of step S102 and step S103. After the backbone application determines the test case, it can perform a scheduled fault-tolerant test and add it to the fault-tolerant monitoring queue to facilitate monitoring of the fault-tolerant test of the backbone application. Since the backbone application and the non-backbone application will also perform the developer's self-testing process after the development is completed and released to the pre-release cluster, the fault-tolerant monitoring queue can include the test tasks of the backbone application and the non-backbone application. The testing of the backbone application and the testing of the non-backbone application can be a fault-tolerant test that is executed regularly, and the test-related information is sent to the pre-release cluster through a request. The test-related information includes the return result of the fault-tolerant test, that is, the test result. The test result can be notified to the relevant personnel in the form of a report. If the backbone application test fails in the test result, the application to be launched can be returned to re-develop the backbone application. If the non-backbone application test fails in the test result, the application to be launched can be returned to re-develop the non-backbone application. If the test result is that the test passes, the application to be launched is published to the cluster of the online system, that is, published to the online cluster. In the online cluster, cluster testing will be performed regardless of whether the application to be launched is a backbone application or a non-backbone application. At the same time, the cluster test also includes heartbeat monitoring of the backbone application. The cluster test can be a fault-tolerant test that is executed regularly. The test result of the cluster test can be returned to the online cluster, and the online cluster can notify the relevant personnel of the returned test result in the form of a report.
[0112] In order to solve the problems existing in the prior art, an embodiment of the present invention provides a fault tolerance test device 600, such as Figure 6 As shown, the device 600 includes:
[0113] The receiving unit 601 is used to determine whether the application to be launched is a backbone application based on the attribute information of the application to be launched;
[0114] A screening unit 602 is configured to screen a test case template corresponding to the type of the application to be launched if the application to be launched is a backbone application;
[0115] A determination unit 603 is used to determine a test parameter corresponding to the application to be launched based on the test function of the application to be launched;
[0116] The testing unit 604 is used to perform a fault tolerance test on the application to be launched based on the test case template and the test parameters, and obtain a test result.
[0117] It should be understood that the manner of implementing the embodiments of the present invention is different from the Figure 1 The method of the illustrated embodiment is the same and will not be repeated here.
[0118] In an implementation of the embodiment of the present invention, the test parameters include input data and a test algorithm corresponding to the application to be launched; the test unit 604 is specifically used to:
[0119] Determine the test case corresponding to the application to be tested according to the test case template, the input data and the test algorithm, and obtain the expected result corresponding to the application to be launched;
[0120] Through the test case, a fault tolerance test is performed on the application to be launched, and an execution result is obtained;
[0121] The test result of the application to be launched is determined according to the execution result and the expected result.
[0122] In another implementation of the embodiment of the present invention, the testing unit 604 is further configured to:
[0123] Determine whether the similarity between the execution result and the expected result reaches a preset threshold;
[0124] If so, the test result is determined to be a passed test; if not, the test result is determined to be a failed test.
[0125] In another implementation of the embodiment of the present invention, the testing unit 604 is further configured to, if the test result is that the test passes, publish the application to be launched to an online cluster, and perform a cluster test on the application to be launched through the online cluster to obtain a cluster test result;
[0126] The device 600 further includes:
[0127] A sending unit is used to send a notification message if the test result is that the test fails, and the notification message includes the test result.
[0128] In another implementation of the embodiment of the present invention, the device 600 further includes:
[0129] A judging unit, used to judge whether the test data result in the cluster test includes a buried point mark;
[0130] A storage unit is used for storing the test data if no; and not storing the test data if yes;.
[0131] In another implementation of the embodiment of the present invention, the device 600 further includes:
[0132] A publishing unit, configured to publish the application to be launched to an online cluster if the application to be launched is not a backbone application;
[0133] The testing unit is further configured to perform a cluster test on the application to be put online through the online cluster to obtain a cluster test result.
[0134] In another implementation manner of the embodiment of the present invention, the attribute information includes an identifier of the application to be launched, a priority, and whether it is a main function.
[0135] It should be understood that the manner of implementing the embodiments of the present invention is different from the Figure 2 or Figure 4 The method of the illustrated embodiment is the same and will not be repeated here.
[0136] In the embodiment of the present invention, the applications to be launched are divided into backbone applications and non-backbone applications. Since the functions implemented by the non-backbone applications do not affect the main functions of the system, fault-tolerance testing is not required and the applications are directly published to the online cluster. That is, only fault-tolerance testing is performed on the backbone applications, thereby reducing the number of applications to be tested. Moreover, the test case template corresponding to the type of the application to be tested can be screened without rewriting the test case for it, thereby saving time for application testing. Even when the number of clusters and the number of service applications are large, the workload of the test task can be reduced and the test efficiency can be improved.
[0137] According to an embodiment of the present invention, the present invention also provides an electronic device and a readable storage medium.
[0138] The electronic device of the present invention comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the one processor, and the instructions are executed by the at least one processor so that the at least one processor executes the fault-tolerant testing method provided in an embodiment of the present invention.
[0139] Figure 7 An exemplary system architecture 700 is shown to which the method for fault-tolerance testing or the apparatus for fault-tolerance testing according to the embodiments of the present invention may be applied.
[0140] like Figure 7 As shown, system architecture 700 may include terminal devices 701, 702, 703, network 704 and server 705. Network 704 is used to provide a medium for communication links between terminal devices 701, 702, 703 and server 705. Network 704 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.
[0141] The user can use the terminal devices 701, 702, 703 to interact with the server 705 through the network 704 to receive or send messages, etc. Various client applications can be installed on the terminal devices 701, 702, 703.
[0142] The terminal devices 701 , 702 , and 703 may be smart phones, tablet computers, laptop computers, desktop computers, and the like.
[0143] The server 705 may be a server that provides various services. The server may analyze and process the received data such as the product information query request, and feed back the processing result (such as product information - just an example) to the terminal device.
[0144] It should be noted that the fault-tolerance testing method provided in the embodiment of the present invention is generally executed by the server 705 , and accordingly, the fault-tolerance testing device is generally disposed in the server 705 .
[0145] It should be understood that Figure 7 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.
[0146] Reference below Figure 8 , which shows a schematic diagram of the structure of a computer system 800 suitable for implementing an embodiment of the present invention. Figure 8 The computer system shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.
[0147] like Figure 8As shown, the computer system 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage part 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the system 800 are also stored. The CPU 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.
[0148] The following components are connected to the I / O interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that a computer program read therefrom is installed into the storage section 808 as needed.
[0149] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the central processing unit (CPU) 801, the above-mentioned functions defined in the system of the present invention are executed.
[0150] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.
[0151] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a unit, a program segment, or a part of a code, and the above-mentioned unit, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0152] The units involved in the embodiments of the present invention may be implemented by software or hardware. The units described may also be set in a processor, for example, it may be described as: a processor includes a receiving unit, a screening unit, a determining unit, and a testing unit. The names of these units do not constitute a limitation on the units themselves in some cases, for example, the receiving unit may also be described as a "unit of the function of the receiving unit".
[0153] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device executes the fault-tolerance test method provided by the present invention.
[0154] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A method for fault tolerance testing, characterized in that: include: Based on the attribute information of the application to be launched, determining whether the application to be launched is a backbone application; If the application to be launched is a backbone application, then according to the type of the application to be launched, a test case template corresponding to the type is selected; Determine, based on the test function of the application to be launched, a test parameter corresponding to the application to be launched; Based on the test case template and the test parameters, perform a fault tolerance test on the application to be launched, and obtain a test result; Also includes: If the application to be launched is not a backbone application, publishing the application to be launched to an online cluster; A cluster test is performed on the application to be put online through the online cluster to obtain a cluster test result.
2. The method according to claim 1, characterized in that: The test parameters include input data and test algorithms corresponding to the application to be launched; The performing a fault tolerance test on the application to be launched based on the test case template and the test parameters, and obtaining a test result, includes: Determine the test case corresponding to the application to be launched according to the test case template, the input data and the test algorithm, and obtain the expected result corresponding to the application to be launched; Through the test case, a fault tolerance test is performed on the application to be launched, and an execution result is obtained; The test result of the application to be launched is determined according to the execution result and the expected result.
3. The method according to claim 2, characterized in that Determining the test result of the application to be launched according to the execution result and the expected result includes: Determine whether the similarity between the execution result and the expected result reaches a preset threshold; If so, the test result is determined to be a passed test; if not, the test result is determined to be a failed test.
4. The method according to claim 1, characterized in that: After performing the fault tolerance test on the application to be launched based on the test case template and the test parameters and obtaining the test result, the method further includes: If the test result is that the test passes, the application to be launched is published to the online cluster, and a cluster test is performed on the application to be launched through the online cluster to obtain a cluster test result; If the test result is that the test fails, a notification message is sent, and the notification message includes the test result.
5. The method according to claim 4, characterized in that After performing cluster testing on the application to be put online through the online cluster and obtaining a cluster testing result, the method further includes: Determine whether the test data in the cluster test includes a point embedding mark; If not, the test data is stored; if so, the test data is not stored.
6. A fault-tolerance testing device, characterized in that: include: A receiving unit, configured to determine whether the application to be launched is a backbone application based on the attribute information of the application to be launched; A screening unit, configured to screen a test case template corresponding to the type of the application to be launched if the application to be launched is a backbone application; A determination unit, configured to determine a test parameter corresponding to the application to be launched based on the test function of the application to be launched; A testing unit, configured to perform a fault-tolerance test on the application to be launched based on the test case template and the test parameters, and obtain a test result; The device also includes: A publishing unit, configured to publish the application to be launched to an online cluster if the application to be launched is not a backbone application; The testing unit is further configured to perform a cluster test on the application to be put online through the online cluster to obtain a cluster test result.
7. The device according to claim 6, characterized in that The test parameters include input data and test algorithms corresponding to the application to be launched; The testing unit is specifically used for: Determine the test case corresponding to the application to be launched according to the test case template, the input data and the test algorithm, and obtain the expected result corresponding to the application to be launched; Through the test case, a fault tolerance test is performed on the application to be launched, and an execution result is obtained; The test result of the application to be launched is determined according to the execution result and the expected result.
8. The device according to claim 7, characterized in that The testing unit is specifically used for: Determine whether the similarity between the execution result and the expected result reaches a preset threshold; If so, the test result is determined to be a passed test; if not, the test result is determined to be a failed test.
9. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.
10. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.
