Parallel distributed database engine testing method and system based on dynamic evaluation
Through dynamic evaluation and parallelized testing methods, the problems of low efficiency, low resource utilization and large comparison errors in distributed database engine testing are solved, efficient and intelligent automated testing is realized, and the test speed and accuracy are improved. And realize integration with other systems.
Patent Information
- Application Number
- CN202111592199.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-23
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-12-23
AI Technical Summary
Traditional distributed database engine functional tests have problems such as slow execution speed, low resource utilization, large results comparison errors and low integration with other systems.
Adopt a parallel distributed database engine test method based on dynamic evaluation. By setting up multiple test clusters, dynamically allocate use case groups to execute in parallel, use object comparison and content filtering rules to improve the accuracy of contrast, and with automated pipeline scheduling system and task management Integration of system and other components.
It significantly improves the testing speed and resource utilization rate, improves the comparison accuracy, and realizes full automation of the test process.
Smart Images

Figure CN114356758B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of distributed database testing, and in particular relates to a parallel distributed database engine testing method and system based on dynamic evaluation. Background Art
[0002] As a distributed database, GBase 8a database has a very large number of execution engine function test case groups. From the perspective of traditional automated testing of GBase 8a distributed database products, there are the following problems:
[0003] 1. Slow execution speed: Due to the extremely large use case group, the traditional physical machine-based serial automated testing has a slow execution speed, which has obvious disadvantages in test scenarios that pursue timeliness, such as regression verification.
[0004] 2. Low resource utilization: Due to the complexity of distributed database functions and the distributed characteristics of deployment, the required test execution environment is relatively complex, and traditional testing methods cannot efficiently utilize computing and storage resources;
[0005] 3. Large error in result comparison: Since traditional testing methods usually use a plain text comparison method based on a database result set, the accuracy of the comparison results is not high and the false alarm rate is high.
[0006] 4. Low integration with other systems: Since traditional black box testing methods are generally not integrated with platforms such as source code management systems, continuous integration systems, task management systems, and email systems, the entire function is relatively isolated and cannot be developed more deeply and comprehensively through automation and intelligent development.
[0007] Based on the above reasons, an efficient and intelligent automated testing method is urgently needed for the execution engine functional testing of the GBase 8a distributed database. Summary of the invention
[0008] In view of this, the present invention aims to propose a parallel distributed database engine testing method and system based on dynamic evaluation to solve the problems of low efficiency, low resource utilization, large error compared with expected results and low integration when performing use case testing on existing databases.
[0009] To achieve the above object, the technical solution of the present invention is achieved as follows:
[0010] On the one hand, this application proposes a parallel distributed database engine testing method based on dynamic evaluation, and the specific steps are as follows:
[0011] S1. The server prepares for the test task, sets up multiple test clusters, determines the use cases to be tested according to the test task, and divides the use cases to be tested into use case groups corresponding to the number of test clusters. Each test cluster tests a group of use case groups to ensure that the total estimated time consumption of the use cases in each group is close. Each use case group dynamically generates execution code for later testing.
[0012] S2. The server formally executes the test task, deploys the database on multiple test clusters in parallel, and executes the execution code of the use case group. The use case execution steps in each test cluster group are: read and execute the use case group, return the result, compare the returned result with the expected result of the use case, obtain the comparison result, record the use case time, and record the information obtained during the execution and comparison of each use case into a corresponding record object;
[0013] S3. After all use cases are executed, the record object in step S25 is imported into the database, and a use case result table is set in the database, which records the information in the record object.
[0014] Furthermore, in step S1, the server performs preparations before the test task, sets up multiple test clusters, determines the use cases to be tested according to the test task, and divides the use cases to be tested into use case groups corresponding to the number of test clusters, ensuring that the total estimated time consumption of the use cases in each group is close, and each use case group dynamically generates execution code for later testing. The specific steps are as follows:
[0015] S11. Query the current server resource surplus and create N test clusters according to the memory usage;
[0016] S12, determining M test cases that need to be executed according to the test task, estimating the historical average time consumption of each of the M test cases that need to be executed, and obtaining the “estimated time consumption” of each test case;
[0017] S13, perform use case grouping calculation, divide the M use cases into N groups of use case groups, and try to ensure that the total estimated time consumption of the use cases in each group is close;
[0018] S14, dynamically generating execution codes for N groups of use cases according to the N groups of use cases generated by grouping in step S13;
[0019] The number N of the test clusters is 2-10, the number of the test clusters is an integer multiple of 2, and the number of use case groups is set corresponding to the number of test clusters.
[0020] Furthermore, in step S12, the historical average time consumption of each use case to be executed is estimated one by one. The specific method is: calculate the average time consumption of each use case during the last 10 executions to obtain the "estimated time consumption" of each use case.
[0021] Furthermore, in step S13, the specific method of grouping the use cases is: selecting one of two grouping algorithms, the dynamic programming method and the rapid distribution method, so as to obtain a sum of estimated time consumption of the use cases in each group of the N groups of use cases close to each other;
[0022] The specific method of the dynamic programming method is to simulate all possible groupings of M use cases and find the optimal solution with the estimated total time consumption of the use cases in each group close to each other;
[0023] The specific method of the fast distribution method is to arrange the estimated installation time of M use cases from large to small, and distribute them to the group with the smallest current estimated installation time in sequence;
[0024] The specific method of selecting one of the two grouping algorithms, dynamic programming and fast distribution, is as follows:
[0025] S131, calculating the time difference between the most time-consuming use case and the least time-consuming use case among the M use cases;
[0026] S132, calculating the ratio Q of the time difference to the total number of use cases M;
[0027] S133, setting Q=2 as a threshold, selecting dynamic programming grouping when Q is higher than 2.0, and selecting rapid distribution grouping when Q is lower than 2.0.
[0028] Further, in step S2, the use case group is read and executed, the result is returned, the returned result is compared with the expected result of the use case, the comparison result is obtained, the use case time is recorded, and the information obtained when each use case is executed and compared is recorded in a corresponding record object. The specific method is as follows:
[0029] S21, read the use case tag pre-added by the author in the use case, start timing, and perform the pre-test preparation work specified in the use case tag, the pre-test preparation work includes adjusting cluster parameters and restarting services;
[0030] S22. Parse the use case content, convert the entire use case from text into a statement object in the program, and parse the customized behavior set in the use case;
[0031] S23, execute the use case statements one by one, and store the returned results temporarily in the memory;
[0032] S24, compare the result returned in step S23 with the expected result of the use case, configure the content filtering rules formulated according to the test requirements, start comparing literal values and data types, and when the comparison is completed, obtain the comparison result, and restore the test cluster according to the customized behavior in step S22;
[0033] S25. End the timing, record the corresponding information of each use case into a corresponding record object, and write the content required by the customer in the use case into a result information file common to the N test clusters.
[0034] Furthermore, in step S133, when Q=2 is set as the threshold, it is obtained by simulation calculation. The specific calculation method of the simulation calculation is as follows:
[0035] S1331. Randomly generate an array of length n, where all elements in the array are positive integers;
[0036] S1332, calculating the Q value of the array in step S1331 and performing group calculations on the array using the dynamic programming method and the fast distribution method, respectively, to obtain three values: the Q value of the array, the variance of the fast distribution result, and the variance of the dynamic programming result;
[0037] S1333. Repeat steps S1331-S1332 for more than 1000 times. According to the statistical results, it is found that when Q is greater than 2, the variance of the dynamic programming algorithm results is generally smaller. Conversely, when Q is less than 2, the variance of the rapid distribution method results is generally smaller. That is, taking Q=2 as the dividing point, when it is higher than 2, the grouping result obtained by the dynamic programming method is better, and when it is lower than 2, the result obtained by the rapid distribution method is better.
[0038] Furthermore, in step S21, the information in the use case tag also includes the author of the use case and the applicable database version;
[0039] In step S23, the customized behavior includes building a database, building a table, and modifying internal parameters;
[0040] In step S24, the use case expected result file is uploaded by the tester when uploading the use case;
[0041] Content filtering rules are to ignore uncontrollable data, including ignoring task numbers and dates;
[0042] In step S25, the recorded content of the record object includes the use case comparison results, failed use cases, use case duration, and use case label information.
[0043] Furthermore, the test task in step S1 is established through the client, which includes the task management platform Redmine; the server includes the pipeline trigger pipeline trigger and the test task scheduling tool Jenkins;
[0044] Redmine monitors and finds that a new test task has been created, and notifies Jenkins to create the test task on the server through the pipeline trigger.
[0045] Furthermore, in step S4, the use case result table will be used to generate a report email and notify the tester. After the tester replies to the email, the content of the reply email will be updated in the database.
[0046] On the other hand, the present application also proposes a parallel distributed database engine testing method system based on dynamic evaluation based on the above-mentioned parallel distributed database engine testing method based on dynamic evaluation.
[0047] Clients include the task management platform Redmine;
[0048] The server is equipped with pipeline trigger, test task scheduling tool Jenkins, test project executor Executor, and mail service module Test Agent;
[0049] Redmine monitors and finds that a new automated test task has been created. It notifies Jenkins to create the test task on the server through the pipeline trigger. Then, the Executor performs the preparation and test tasks before the test task.
[0050] Test Agent is responsible for sending the generated report to the tester after the test is completed. The tester replies to the failed use case analysis email, and Test Agent will update the tester's reply email to the database. Test Agent also cooperates with Executor to obtain the estimated time consumption of each use case historical execution record.
[0051] Compared with the prior art, the parallel distributed database engine testing method and system based on dynamic evaluation described in the present invention has the following beneficial effects:
[0052] (1) The parallel distributed database engine testing method and system based on dynamic evaluation described in the present invention estimates the resource margin for testing and the time consumption of each use case before the test begins, and adopts a suitable use case distribution method as much as possible to ensure that a large number of use cases are evenly distributed to as many test clusters as possible according to the time consumption, so that the overall test speed is increased exponentially.
[0053] (2) The parallel distributed database engine testing method and system based on dynamic evaluation described in the present invention adopts object-based comparison rather than string comparison for comparison results, so that the framework can also accurately compare complex data types; add content filtering rules to filter dynamic values (such as time, task number, etc., which will inevitably be different in each test), so that the accuracy of the entire comparison process is greatly improved;
[0054] (3) The parallel distributed database engine testing method and system based on dynamic evaluation described in the present invention improve the full automation of the testing process. This testing method is designed based on a cloud native platform. Each component is deployed in the form of a service and cooperates with components such as an automated assembly line scheduling system, a task management system, and an email system to achieve full automation of the entire operation process. After the test task is released on the task management platform, the tester only needs to pay attention to the test report email without performing any manual operation. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:
[0056] Figure 1 A schematic diagram of a parallel distributed database engine testing method and system based on dynamic evaluation according to an embodiment of the present invention;
[0057] Figure 2 This is a schematic diagram of the use case grouping principle described in an embodiment of the present invention. DETAILED DESCRIPTION
[0058] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.
[0059] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and the like are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", and the like may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.
[0060] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood by specific circumstances.
[0061] The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.
[0062] like Figure 1 As shown in the figure, the parallel distributed database engine testing method based on dynamic evaluation has the following specific steps:
[0063] S1. The server prepares for the test task, sets up multiple test clusters, determines the use cases to be tested according to the test task, and divides the use cases to be tested into use case groups corresponding to the number of test clusters. Each test cluster tests a group of use case groups to ensure that the total estimated time consumption of the use cases in each group is close. Each use case group dynamically generates execution code for later testing.
[0064] S2. The server formally executes the test task, deploys the database on multiple test clusters in parallel, and executes the execution code of the use case group. The use case execution steps in each test cluster group are: read and execute the use case group, return the result, compare the returned result with the expected result of the use case, obtain the comparison result, record the use case time, and record the information obtained during the execution and comparison of each use case into a corresponding record object;
[0065] S3. After all use cases are executed, the record object in step S25 is imported into the database, and a use case result table is set in the database, which records the information in the record object.
[0066] After the test is completed, the use case result table generates a report email and sends it to the tester, who will reply to the failed use case analysis email;
[0067] The use case result table will serve as the basis for subsequent use case data statistics, report generation, use case distribution calculation and other operations.
[0068] like Figure 1 As shown, in step S1, the server prepares for the test task, sets up multiple test clusters, determines the use cases to be tested according to the test task, and divides the use cases to be tested into use case groups corresponding to the number of test clusters, ensuring that the total estimated time consumption of the use cases in each group is close, and each use case group dynamically generates execution code for later testing. The specific steps are as follows:
[0069] S11. Query the current server resource surplus and create N test clusters according to the memory usage;
[0070] S12, determining M test cases that need to be executed according to the test task, estimating the historical average time consumption of each of the M test cases that need to be executed, and obtaining the “estimated time consumption” of each test case;
[0071] S13, perform use case grouping calculation, divide the M use cases into N groups of use case groups, and try to ensure that the total estimated time consumption of the use cases in each group is close;
[0072] S14, dynamically generating execution codes for N groups of use cases according to the N groups of use cases generated by grouping in step S23;
[0073] The number N of the test clusters is 2-10, the number of the test clusters is an integer multiple of 2, and the number of use case groups is set corresponding to the number of test clusters.
[0074] like Figure 1 As shown, in step S12, the historical average time consumption of the use cases to be executed is estimated one by one. The specific method is: calculate the average time consumption of each use case during the last 10 executions to obtain the "estimated time consumption" of each use case.
[0075] like Figure 1 As shown, in step S13, the specific method of grouping use cases is: selecting one of two grouping algorithms, dynamic programming method and rapid distribution method, to obtain a sum of estimated time consumption of use cases in each group of N groups of use cases close to each other;
[0076] The specific method of the dynamic programming method is to simulate all possible groupings of M use cases and find the optimal solution with the estimated total time consumption of the use cases in each group close to each other;
[0077] The specific method of the fast distribution method is to arrange the estimated installation time of M use cases from large to small, and distribute them to the group with the smallest current estimated installation time in sequence;
[0078] The selection method of choosing one of the two grouping algorithms, dynamic programming and fast distribution, is:
[0079] S131, calculating the time difference between the most time-consuming use case and the least time-consuming use case among the M use cases;
[0080] S132, calculating the ratio Q of the time difference to the total number of use cases M;
[0081] S133, setting Q=2 as a threshold, selecting dynamic programming grouping when Q is higher than 2.0, and selecting rapid distribution grouping when Q is lower than 2.0.
[0082] like Figure 1 , Figure 2As shown, in step S2, read and execute the use case group, return the result, compare the returned result with the expected result of the use case, obtain the comparison result, record the use case time, and record the information obtained when each use case is executed and compared into a corresponding record object. The specific method is as follows:
[0083] S21, read the use case tag pre-added by the author in the use case, start timing, and perform the pre-test preparation work specified in the use case tag, the pre-test preparation work includes adjusting cluster parameters and restarting services;
[0084] S22. Parse the use case content, convert the entire use case from text into a statement object in the program, and parse the customized behavior set in the use case;
[0085] S23, execute the use case statements one by one, and store the returned results temporarily in the memory;
[0086] S24, compare the result returned in step S23 with the expected result of the use case, configure the content filtering rules formulated according to the test requirements, start comparing literal values and data types, and when the comparison is completed, obtain the comparison result, and restore the test cluster according to the customized behavior in step S22;
[0087] S25. End the timing, record the corresponding information of each use case into a corresponding record object, and write the content required by the customer in the use case into a result information file common to the N test clusters.
[0088] like Figure 1 , Figure 2 As shown, in step S133, when Q=2 is set as the threshold, it is obtained by simulation calculation. The specific calculation method of the simulation calculation is as follows:
[0089] S1331. Randomly generate an array of length n, where all elements in the array are positive integers;
[0090] S1332, calculating the Q value of the array in step S1331 and performing group calculations on the array using the dynamic programming method and the fast distribution method, respectively, to obtain three values: the Q value of the array, the variance of the fast distribution result, and the variance of the dynamic programming result;
[0091] S1333. Repeat steps S1331-S1332 for more than 1000 times. According to the statistical results, it is found that when Q is greater than 2, the variance of the dynamic programming algorithm results is generally smaller. Conversely, when Q is less than 2, the variance of the rapid distribution method results is generally smaller. That is, taking Q=2 as the dividing point, when it is higher than 2, the grouping result obtained by the dynamic programming method is better, and when it is lower than 2, the result obtained by the rapid distribution method is better.
[0092] Since the estimated time consumption of a use case is often very close to the actual time consumption, the above method is used to calculate the theoretically optimal grouping method, which can also greatly improve the actual execution speed. For example, if it takes 4 hours to execute a test using 2 clusters, the time consumption can be shortened to 2 hours using 4 clusters, thereby improving test efficiency.
[0093] like Figure 1 As shown, in step S21, the information in the use case tag also includes the author of the use case and the applicable database version;
[0094] In step S23, the customized behavior includes building a database, building a table, and modifying internal parameters;
[0095] In step S24, the use case expected result file is uploaded by the tester when uploading the use case;
[0096] Content filtering rules are to ignore uncontrollable data, including ignoring task numbers and dates;
[0097] In step S25, the recorded content of the record object includes the use case comparison results, failed use cases, use case duration, and use case label information.
[0098] The test task in step S1 is established through the client, which includes the task management platform Redmine; the server includes the pipeline trigger and the test task scheduling tool Jenkins;
[0099] Redmine monitors and finds that a new test task has been created, and notifies Jenkins to create the test task on the server through the pipeline trigger.
[0100] In step S3, the use case result table will be used to generate a report email and notify the tester. After the tester replies to the email, the content of the reply email will be updated in the database.
[0101] like Figure 1 As shown, on the other hand, the present application also proposes a parallel distributed database engine testing method system based on dynamic evaluation based on the above-mentioned parallel distributed database engine testing method based on dynamic evaluation.
[0102] Clients include the task management platform Redmine;
[0103] The server is equipped with pipeline trigger, test task scheduling tool Jenkins, test project executor Executor, and mail service module Test Agent;
[0104] Redmine monitors and finds that a new automated test task has been created. It notifies Jenkins to create the test task on the server through the pipeline trigger. Then, the Executor performs the preparation and test tasks before the test task.
[0105] Test Agent is responsible for sending the generated report to the tester after the test is completed. The tester replies to the failed use case analysis email, and Test Agent will update the tester's reply email to the database. Test Agent also cooperates with Executor to obtain the estimated time consumption of each use case historical execution record.
[0106] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A parallel distributed database engine testing method based on dynamic evaluation, characterized in that: The specific steps are as follows: S1. The server prepares for the test task, sets up multiple test clusters, determines the use cases to be tested according to the test task, and divides the use cases to be tested into use case groups corresponding to the number of test clusters. Each test cluster tests a group of use case groups to ensure that the total estimated time consumption of the use cases in each group is close. Each use case group dynamically generates execution code for later testing. Estimate the historical average time consumption of each use case to be executed. The specific method is: calculate the average time consumption of each use case during the last 10 executions to obtain the "estimated time consumption" of each use case; Select one of the two grouping algorithms, dynamic programming and rapid distribution, to obtain a sum of estimated time consumption of the use cases in each of the N groups of use cases close to each other; The specific method of dynamic programming is to simulate all the groups of M use cases and find the optimal solution with the estimated time sum of the use cases in each group close to each other; The specific method of the fast distribution method is to arrange the estimated installation time of M use cases from large to small, and distribute them to the group with the smallest estimated installation time in sequence; The specific method of selecting one of the two grouping algorithms, dynamic programming and fast distribution, is as follows: S131, calculating the time difference between the most time-consuming use case and the least time-consuming use case among the M use cases; S132, calculating the ratio Q of the time difference to the total number of use cases M; S133, set Q=2 as the threshold, select dynamic programming method for grouping when Q is higher than 2.0, and select rapid distribution method for grouping when Q is lower than 2.0; In step S133, Q=2 is set as the threshold value and is obtained through simulation calculation. The specific calculation method of the simulation calculation is as follows: S1331. Randomly generate an array of length n, where all elements in the array are positive integers; S1332, calculating the Q value of the array in step S1331 and performing group calculations on the array using the dynamic programming method and the fast distribution method, respectively, to obtain three values: the Q value of the array, the variance of the fast distribution result, and the variance of the dynamic programming result; S1333, repeating steps S1331-S1332 for more than 1000 times, and according to the statistical results, it is found that when Q is greater than 2, the variance of the dynamic programming algorithm result is smaller, and vice versa, when Q is less than 2, the variance of the rapid distribution method result is smaller, that is, taking Q = 2 as the dividing point, when it is higher than 2, the grouping result obtained by the dynamic programming method is better, and when it is lower than 2, the result obtained by the rapid distribution method is better; S2. The server formally executes the test task, deploys the database on multiple test clusters in parallel, and executes the execution code of the use case group. The use case execution steps in each test cluster group are: read and execute the use case group, return the result, compare the returned result with the expected result of the use case, obtain the comparison result, record the use case time, and record the information obtained during the execution and comparison of each use case into a corresponding record object; S3. After all use cases are executed, the record object in step S25 is imported into the database, and a use case result table is set in the database, which records the information in the record object.
2. The parallel distributed database engine testing method based on dynamic evaluation according to claim 1, characterized in that: In step S1, the server prepares for the test task, sets up multiple test clusters, determines the use cases to be tested according to the test task, and divides the use cases to be tested into use case groups corresponding to the number of test clusters, ensuring that the total estimated time consumption of the use cases in each group is close, and each use case group dynamically generates execution code for later testing. The specific steps are as follows: S11. Query the current server resource surplus and create N test clusters according to the memory usage; S12, determining M test cases that need to be executed according to the test task, estimating the historical average time consumption of each of the M test cases that need to be executed, and obtaining the "estimated time consumption" of each test case; S13, perform use case grouping calculation, divide the M use cases into N groups of use case groups, and try to ensure that the total estimated time consumption of the use cases in each group is close; S14, dynamically generating execution codes for N groups of use cases according to the N groups of use cases generated by grouping in step S13; The number N of the test clusters is 2-10, the number of the test clusters is an integer multiple of 2, and the number of use case groups is set corresponding to the number of test clusters.
3. The parallel distributed database engine testing method based on dynamic evaluation according to claim 2, characterized in that: In step S2, read and execute the use case group, return the result, compare the returned result with the expected result of the use case, obtain the comparison result, record the use case time, and record the information obtained when each use case is executed and compared into a corresponding record object. The specific method is as follows: S21, read the use case tag pre-added by the author in the use case, start timing, and perform the pre-test preparation work specified in the use case tag, the pre-test preparation work includes adjusting cluster parameters and restarting services; S22. Parse the use case content, convert the entire use case from text into a statement object in the program, and parse the customized behavior set in the use case; S23, execute the use case statements one by one, and store the returned results temporarily in the memory; S24, compare the result returned in step S23 with the expected result of the use case, configure the content filtering rules formulated according to the test requirements, start comparing literal values and data types, and when the comparison is completed, obtain the comparison result, and restore the test cluster according to the customized behavior in step S22; S25. End the timing, record the corresponding information of each use case into a corresponding record object, and write the content required by the customer in the use case into a result information file common to the N test clusters.
4. The parallel distributed database engine testing method based on dynamic evaluation according to claim 3, characterized in that: In step S21, the information in the use case tag also includes the author of the use case and the applicable database version; In step S23, the customized behavior includes building a database, building a table, and modifying internal parameters; In step S24, the use case expected result file is uploaded by the tester when uploading the use case; Content filtering rules include ignoring uncontrollable data, ignoring task numbers, and ignoring dates; In step S25, the recorded object records content including use case comparison results, failed use cases, use case duration, and use case label information.
5. The parallel distributed database engine testing method based on dynamic evaluation according to claim 1, characterized in that: The test task in step S1 is established through the client, which includes the task management platform Redmine; the server includes the pipeline trigger and the test task scheduling tool Jenkins; Redmine monitors and finds that a new test task has been created, and notifies Jenkins to create the test task on the server through the pipeline trigger.
6. The parallel distributed database engine testing method based on dynamic evaluation according to claim 1, characterized in that: In step S3, the use case result table will be used to generate a report email and notify the tester. After the tester replies to the email, the content of the reply email will be updated in the database.
7. A parallel distributed database engine testing method system based on dynamic evaluation based on a dynamic evaluation method according to any one of claims 1 to 6, characterized in that: Clients include the task management platform Redmine; The server is equipped with pipeline trigger, test task scheduling tool Jenkins, test project executor Executor, and mail service module Test Agent; Redmine monitors and finds that a new automated test task has been created. It notifies Jenkins to create the test task on the server through the pipeline trigger. Then, the Executor performs the preparation and test tasks before the test task. Test Agent is responsible for sending the generated report to the tester after the test is completed. The tester replies to the failed use case analysis email, and Test Agent will update the tester's reply email to the database. Test Agent also cooperates with Executor to obtain the estimated time consumption of each use case historical execution record.
Citation Information
Patent Citations
Database based automation parallel test platform execution method
CN105512038A
Database performance test method and device, electronic equipment and storage medium
CN113127312A
