A distributed performance testing method based on jmeter

Abstract

The present invention relates to a distributed performance testing method based on JMeter, comprising the following steps: S1: batch generating scheduled task execution times through Excel formulas; S2: setting multiple stress testing scenarios expected to be executed based on JMeter's JMX script according to the parameters of the stress testing scenarios; S3: traversing the JMX scripts to be executed, obtaining the JMX script names and the total number; S4: calculating the number of concurrent executions required for each JMeter press machine according to the parameters of the stress testing scenarios, cyclically generating execution commands and writing them to a file; S5: combining the execution times of existing scheduled tasks in Excel, batch splicing the execution commands in the files to generate a spliced ​​command file; S6: uploading the spliced ​​command file to the JMeter press machine, executing the commands to deploy the scheduled tasks; and S7: generating a JTL test result file named according to a rule. The present invention can complete the automated performance testing process and improve test efficiency when the JMeter press machine does not meet the required pressure.

Description

Technical Field

[0001] The present invention relates to the technical field of software performance testing, and in particular to a distributed performance testing method based on JMeter. Background Art

[0002] The existing literature on performance testing methods based on Jmeter is as follows: Literature 1 (CN116383025A) proposes an execution method based on Jenkins job scheduling to trigger Jmeter scripts; Literature 2 (CN110417613A) proposes a Jmeter distributed execution scheduling method; Literature 3 (CN108540349A) proposes a single-client execution method and result collection method.

[0003] The existing technology has the following defects: (1) it cannot complete the automated performance testing process and improve the testing efficiency when a single Jmeter client does not meet the required pressure; (2) it does not involve multi-scenario execution strategies under multiple Jmeter clients; (3) it does not involve the merging of test execution report data from multiple Jmeter clients; and (4) it does not involve the automatic interception of test execution results.

[0004] Therefore, it is necessary to provide a distributed performance testing method based on jmeter. When the execution of the jmeter press does not meet the required pressure, the pressure concurrency can be increased by deploying multiple jmeter presses to complete the automated performance testing process and improve the testing efficiency. Summary of the Invention

[0005] The purpose of the present invention is to provide a distributed performance testing method based on jmeter, so that when a single jmeter press machine cannot meet the required pressure, the pressure concurrency can be increased by deploying multiple jmeter press machines to complete the automated performance testing process and improve the testing efficiency.

[0006] In order to solve the problems existing in the prior art, the present invention provides a distributed performance testing method based on JMeter, comprising the following steps:

[0007] Step 1: Generate scheduled task execution time in batches through Excel formulas;

[0008] Step 2: Based on the JMeter JMX script, set multiple stress testing scenarios that you expect to execute according to the parameters of the stress testing scenario. The JMX script corresponds to the stress testing scenario one by one.

[0009] Step 3: Traverse the jmx scripts that need to be executed and obtain the jmx script names and total number;

[0010] Step 4: Calculate the number of concurrent executions of the stress test scenario required by each JMeter stress machine based on the stress test scenario parameters, cyclically generate execution commands for executing the JMX script, and write the execution commands to a file;

[0011] Step 5: Combine the existing scheduled task execution time in Excel, execute commands in batches of spliced ​​files, and generate a spliced ​​command file;

[0012] Step 6: Upload the spliced ​​command file to the jmeter press, and execute the scheduled task deployed by the command in the spliced ​​command file on the jmeter press;

[0013] step7: Generate a JTL test result file named according to the rules.

[0014] Optionally, in the jmeter-based distributed performance testing method, the stress testing scenario includes three parameters, namely: the total number of concurrency in different scenarios, the number of stress machines and the number of thread groups actually executed by the script settings.

[0015] Optionally, the jmeter-based distributed performance testing method also includes the following steps: step 8: merging the test results of multiple jmeter presses and generating test results that meet the requirements. Step 8 includes JTL test result file preprocessing, file merging, aggregate report generation and report information enhancement.

[0016] Optionally, in the jmeter-based distributed performance testing method, in the JTL test result file preprocessing stage, first extract the names of all JTL test result files to be executed in the directory and store them in the jtlList file, and then process the JTL test result files in the following manner: check whether the connection time of the JTL test result file is empty, if it is empty, delete the first and last two lines of the JTL test result file, if it is not empty, delete the first line; the processed JTL test result file is renamed to SRC_source file name;

[0017] In the file merging stage, first obtain the scenario name from the scenario list file, create a new JTL file, and write the header. Then, based on the scenario name, append the contents of all JTL test result files named with SRC_source file name to the new JTL file.

[0018] In the aggregate report generation stage, use the jpgc-cmd-simple component in the JMeter plug-in to generate an aggregate report by executing the JMeterPluginsCMD.sh command and perform data cleaning on the generated CSV file;

[0019] In the report information enhancement stage, first extract the time information, extract the maximum and minimum timestamps from the new JTL file and convert them into a readable time format, parse the scenario name and number of threads from the new JTL file name, append the scenario start time, end time, number of threads and scenario name to the first line of the CSV file, save the information to the senceTime.txt file, and update the header of the CSV file.

[0020] Optionally, the jmeter-based distributed performance testing method further includes the following steps: Step 9: Automatically take a screenshot of the required test results and automatically save them in a word document.

[0021] Optionally, in the jmeter-based distributed performance testing method, the screenshot and preservation method is as follows: generate a blank word document through the apache toolkit, read the scenario list file in a loop, use the JMeterPluginsCMD plug-in manager to generate a trend chart scenario of response time changing with test time, and use the JMeterPluginsCMD plug-in manager to generate a trend chart of throughput changing with test time; use the Apache POI tool to add the scenario name as the document subtitle, and add the trend chart scenario and trend chart to the blank word document.

[0022] Optionally, the jmeter-based distributed performance testing method further includes the following steps: step 10: automatically generating a resource monitoring excel file based on the test execution start time and end time in the sentenceTime.txt file.

[0023] Optionally, in the jmeter-based distributed performance testing method, a resource monitoring Excel file is generated as follows: obtain the test execution time, obtain resource usage data through the Prometheus API interface, and write it into Excel.

[0024] Optionally, in the jmeter-based distributed performance testing method, the resource usage data includes average CPU usage, average memory usage, and disk I / O busy rate.

[0025] Compared with the prior art, the present invention has the following advantages:

[0026] (1) When a single jmeter press machine does not meet the required pressure, the present invention increases the pressure concurrency by deploying multiple jmeter press machines, completes the automated performance test process and improves the test efficiency, thereby reducing manual operations and reducing the overall time consumption of non-power testing.

[0027] (2) In the JTL merging stage, the present invention first obtains the scene name from the scene list file, and there is no limit on the number of files to be merged.

[0028] (3) The present invention supports multiple jmeter presses as performance test clients, and when multiple test scenarios need to be executed, it can quickly complete the test process and automatically generate the process indicator data related content required for the test report.

[0029] (4) The performance test scenario of the present invention automatically generates a scheduled task scenario and automatically executes it. When multiple jmeter press machines are pressed at the same time, after execution, the distributed performance test data is automatically merged, the report screenshot is automatically generated and saved to the specified word document, and the application resource usage data is automatically generated and stored in an excel file. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a flow chart of a distributed performance testing method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The following is a more detailed description of the specific embodiments of the present invention with reference to schematic diagrams. The advantages and features of the present invention will become more apparent from the following description. It should be noted that the drawings are greatly simplified and not to exact scale, and are only used for the purpose of conveniently and clearly illustrating the embodiments of the present invention.

[0032] Hereinafter, if the method described herein includes a series of steps, the order in which the steps are presented herein is not necessarily the only order in which the steps may be performed, and some of the steps described may be omitted and / or some other steps not described herein may be added to the method.

[0033] The existing technology has the following defects: (1) it cannot complete the automated performance testing process and improve the testing efficiency when a single Jmeter client does not meet the required pressure; (2) it does not involve multi-scenario execution strategies under multiple Jmeter clients; (3) it does not involve the merging of test execution report data from multiple Jmeter clients; and (4) it does not involve the automatic interception of test execution results.

[0034] In order to solve the problems existing in the prior art, the present invention provides a distributed performance testing method based on JMeter. This method automatically generates scheduled tasks that need to be distributedly executed in multiple scenarios through Excel and shell scripts when multiple JMeter stressors are executed; the test results of the JMeter stressors are automatically merged to generate execution summary data in the form of scripts; the execution result pictures are automatically screenshoted in the form of scripts and stored in a Word document, and the application resource usage data is automatically generated and stored in an Excel file.

[0035] like Figure 1 As shown, the distributed performance testing method based on jmeter includes the following steps:

[0036] Step 1: Generate scheduled task execution time in batches through Excel formulas;

[0037] Step 2: Based on the JMeter JMX script, set multiple stress testing scenarios that you expect to execute according to the parameters of the stress testing scenario. The JMX script corresponds to the stress testing scenario one by one. The stress testing scenario includes three parameters: the total number of concurrency in different scenarios, the number of stress machines actually executed by the script, and the number of thread groups.

[0038] Step 3: Traverse the jmx scripts that need to be executed and obtain the jmx script names and total number;

[0039] Step 4: Calculate the number of concurrent executions of the stress test scenario required by each JMeter stress machine based on the stress test scenario parameters, cyclically generate execution commands for executing the JMX script, and write the execution commands to a file;

[0040] Step 5: Combine the existing scheduled task execution time in Excel, execute commands in batches of spliced ​​files, and generate a spliced ​​command file;

[0041] Step 6: Upload the spliced ​​command file to the jmeter press, and execute the scheduled task deployed by the command in the spliced ​​command file on the jmeter press;

[0042] step7: Generate a JTL test result file named according to the rules.

[0043] Step 8: Merge the test results of multiple JMeter presses and generate test results that meet the requirements. Step 8 includes JTL test result file preprocessing, file merging, aggregate report generation and report information enhancement.

[0044] Specifically, in the JTL test result file preprocessing stage, first extract the names of all JTL test result files to be executed in the directory and save them in the jtlList file, and then process the JTL test result files in the following way: check whether the Connect connection time (located in the 11th column of the last row) of the JTL test result file is empty. If it is empty, delete the first and last two rows of the JTL test result file (the header and invalid data). If it is not empty, delete the first row (the header); rename the processed JTL test result file to SRC_source file name, and delete the original file;

[0045] In the file merging stage, first get the scene name from the scene list file sceneList, create a new JTL file (for example: M_scene name.jtl), and write the header. According to the scene name, all JTL test result files named with SRC_source file name are appended to the new JTL file.

[0046] In the aggregate report generation stage, the jpgc-cmd-simple component in the JMeter plug-in is used to generate an aggregate report by executing the JMeterPluginsCMD.sh command, and the generated CSV file is cleaned. The cleaning processing steps include deleting the table header, deleting the TOTAL row and the Chinese TOTAL row (to avoid duplicate statistics), and sorting the report by the sampler name. Then, each sampler in the report is traversed, and its total number of requests and failures are counted according to the new JTL file. The failure rate is calculated and updated to columns 10 to 12 of the report. At the same time, a summary row is added to calculate the total response time of all samplers, and a TOTAL row is added to the end of the report.

[0047] In the report information enhancement stage, first extract the time information, extract the maximum and minimum timestamps from the new JTL file and convert them into a readable time format (for example: yyyy-MM-dd HH:mm:ss), parse the scenario name and number of threads from the new JTL file name, append the scenario start time, end time, number of threads, and scenario name to the first line of the CSV file, and save the information to the sceneTime.txt file. Update the header of the CSV file to use a more understandable name.

[0048] Step 9: Automatically take screenshots of the required test results and save them in a Word document.

[0049] Specifically, the screenshot and save methods are as follows: generate a blank word document through the Apache toolkit, loop through the scene list file sceneLis, use the JMeterPluginsCMD plug-in manager to generate a trend chart scenario of response time versus test time, with the naming rule being scene name_ART.png, and use the JMeterPluginsCMD plug-in manager to generate a trend chart of throughput versus test time, with the naming rule being scene name_TPS.png; use the Apache POI tool to add the scene name as the document subtitle, and add the trend chart scene (ie, scene name_ART.png) and the trend chart (ie, scene name_TPS.png) to the blank word document.

[0050] Step 10: Automatically generate a resource monitoring Excel file based on the test execution start and end times in the sentenceTime.txt file. Generate the resource monitoring Excel file as follows: obtain the test execution time, obtain resource usage data through the Prometheus API, and write it to Excel. Resource usage data includes average CPU usage, average memory usage, and disk I / O busy rate.

[0051] In one embodiment, a resource monitoring Excel file is generated as follows: first, a Prometheus dashboard template is customized according to the actual scenario, and monitoring applications are grouped and classified by job; a blank Excel document is generated using the Apache POI tool; the start time and end time in the sentenceTime.txt file are obtained as initialization values ​​for the JMeter data extraction script, and the time difference is calculated based on the start time and end time. If the time difference is greater than 2 minutes, the start time and end time are both moved right and left by 1 minute. If the time difference is less than 2 minutes, no processing is performed and the calculation is performed directly. Call the Prometheus API to obtain the average CPU usage of applications on different instances during the interval, calculated as (1 - average CPU idle rate) * 100%. Extract the application name, instance name, and average CPU usage from the returned results. Call the Prometheus API to obtain the average memory usage of applications on different instances during the interval, calculated as (total memory - free memory - buffers - cache) / total memory * 100%. Extract the instance name and average memory usage from the returned results. Call the Prometheus API to obtain the peak disk I / O busy rate of applications on different instances during the interval, calculated as disk I / O processing time / total time * 100%. Extract the instance name and disk I / O busy rate from the returned results. Based on the application and IP address information, combine the application name and instance IP address and sort them. Based on the sorted IP addresses, obtain the average CPU usage, average memory usage, and disk I / O busy rate. Write the query time interval, application name and IP address, average CPU usage, average memory usage, and disk I / O busy rate to the corresponding columns in Excel.

[0052] Compared with the prior art, the present invention has the following advantages:

[0053] (1) When a single jmeter press machine does not meet the required pressure, the present invention increases the pressure concurrency by deploying multiple jmeter press machines, completes the automated performance test process and improves the test efficiency, thereby reducing manual operations and reducing the overall time consumption of non-power testing.

[0054] (2) In the JTL merging stage, the present invention first obtains the scene name from the scene list file, and there is no limit on the number of files to be merged.

[0055] (3) The present invention supports multiple jmeter presses as performance test clients, and when multiple test scenarios need to be executed, it can quickly complete the test process and automatically generate the process indicator data related content required for the test report.

[0056] (4) The performance test scenario of the present invention automatically generates a scheduled task scenario and automatically executes it. When multiple jmeter press machines are pressed at the same time, after execution, the distributed performance test data is automatically merged, the report screenshot is automatically generated and saved to the specified word document, and the application resource usage data is automatically generated and stored in an excel file.

[0057] The above description is merely a preferred embodiment of the present invention and does not limit the present invention in any way. Any person skilled in the art who, without departing from the scope of the present invention, makes any equivalent substitution, modification, or other changes to the technical solution and technical content disclosed in the present invention shall be deemed to be within the scope of the present invention and still fall within the scope of protection of the present invention.

Claims

1. A distributed performance testing method based on jmeter, characterized in that: The following steps are involved: Step 1: Generate scheduled task execution time in batches through Excel formulas; Step 2: Based on the JMeter JMX script, set multiple stress testing scenarios that you expect to execute according to the parameters of the stress testing scenario. The JMX script corresponds to the stress testing scenario one by one. Step 3: Traverse the jmx scripts that need to be executed and obtain the jmx script names and total number; Step 4: Calculate the number of concurrent executions of the stress test scenario required by each JMeter stress machine based on the stress test scenario parameters, cyclically generate execution commands for executing the JMX script, and write the execution commands to a file; Step 5: Combine the existing scheduled task execution time in Excel, execute commands in batches of spliced ​​files, and generate a spliced ​​command file; Step 6: Upload the spliced ​​command file to the jmeter press, and execute the scheduled task deployed by the command in the spliced ​​command file on the jmeter press; Step 7: Generate JTL test result files named according to the rules; Step 8: Merge the test results of multiple JMeter presses and generate test results that meet the requirements. Step 8 includes JTL test result file preprocessing, file merging, aggregate report generation and report information enhancement; Step 9: Automatically take screenshots of the required test results and save them in a Word document; Step 10: Automatically generate a resource monitoring Excel file based on the test execution start time and end time in the sentenceTime.txt file.

2. The distributed performance testing method based on jmeter according to claim 1, characterized in that: The stress testing scenario includes three parameters: the total number of concurrent users in different scenarios, the number of stress machines and the number of thread groups actually executed by the script settings.

3. The distributed performance testing method based on jmeter according to claim 1, characterized in that: In step 8, In the JTL test result file preprocessing stage, first extract the names of all the JTL test result files to be executed in the directory and save them in the jtlList file, then process the JTL test result files as follows: check whether the connection time of the JTL test result file is empty. If it is empty, delete the first and last two lines of the JTL test result file. If it is not empty, delete the first line; the processed JTL test result file is renamed to SRC_source file name; In the file merging stage, first obtain the scenario name from the scenario list file, create a new JTL file, and write the header. Then, based on the scenario name, append the contents of all JTL test result files named with SRC_source file name to the new JTL file. In the aggregate report generation stage, use the jpgc-cmd-simple component in the JMeter plug-in to generate an aggregate report by executing the JMeterPluginsCMD.sh command and perform data cleaning on the generated CSV file; In the report information enhancement stage, first extract the time information, extract the maximum and minimum timestamps from the new JTL file and convert them into a readable time format, parse the scenario name and number of threads from the new JTL file name, append the scenario start time, end time, number of threads and scenario name to the first line of the CSV file, save the information to the senceTime.txt file, and update the header of the CSV file.

4. The distributed performance testing method based on jmeter according to claim 3, characterized in that: In step 9, the method of automatically taking screenshots and automatically saving them is as follows: use the Apache toolkit to generate a blank Word document, loop through the scenario list file, use the JMeterPluginsCMD plug-in manager to generate a trend chart scenario of response time changing with test time, and use the JMeterPluginsCMD plug-in manager to generate a trend chart of throughput changing with test time; use the ApachePOI tool to add the scenario name as the document subtitle, and add the trend chart scenario and trend chart to the blank Word document.

5. The distributed performance testing method based on jmeter according to claim 3, characterized in that: In step 10, the resource monitoring Excel file is generated as follows: obtain the test execution time, obtain resource usage data through the Prometheus API interface, and write it to Excel.

6. The distributed performance testing method based on jmeter according to claim 5, characterized in that: Resource usage data includes average CPU usage, average memory usage, and disk I / O busy rate.

Images