Large concurrency performance test platform implementation method based on log file analysis

Abstract

The invention discloses a large concurrency performance test platform implementation method based on log file analysis, and relates to the field of software testing. A large concurrency performance test platform is established, Jmeter performance test scripts are uploaded, stored and managed through a script manager module, the specified Jmeter performance test scripts are distributed to all Jmeter nodes through a child node manager module, starting and stopping of the Jmeter nodes are controlled, Jmeter tools of the Jmeter nodes are controlled through the child node manager module to execute the performance test scripts, and the Jmeter performance test scripts are distributed to the Jmeter nodes. The method comprises the following steps: generating a test log, asynchronously analyzing the test log by utilizing FileBeat of a Jmeter node, submitting an analyzed log file to a Kafka message queue, forwarding the submitted log file to a report generation component through the Kafka message queue, analyzing a parameter field in the log file through the report generation component, and generating a performance test report according to the parameter field. And displaying the generated performance test report through a display module.

Description

Technical Field

[0001] The invention discloses a method for realizing a large concurrent performance testing platform based on log file analysis, and relates to the field of software testing. Background Art

[0002] In the context of the rapid development of current application technologies, performance testing faces many challenges, especially when simulating large-scale concurrent scenarios, such as testing scenarios with millions of concurrent users. Existing commercial testing tools have some bottlenecks when testing large-scale concurrent users, especially in cluster mode, where the performance scheduling and communication bandwidth of the central control node are constrained. Summary of the invention

[0003] In view of the problems of the prior art, the present invention provides an implementation method of a large-concurrency performance testing platform based on log file parsing. Through the implementation method, the resource requirements in large-concurrency testing are significantly reduced, and the performance testing accuracy is greatly improved. The platform is established to utilize efficient log parsing technology to accurately capture performance data and ensure that the test results are accurate.

[0004] The specific scheme proposed by the present invention is:

[0005] A method for implementing a large concurrent performance test platform based on log file parsing is used to establish a large concurrent performance test platform, wherein the large concurrent performance test platform includes a script manager module, a subnode manager module, a report generation component and a display module.

[0006] Upload, store and manage Jmeter performance test scripts through the script manager module, distribute the specified Jmeter performance test scripts to all Jmeter nodes through the subnode manager module, and control the start and stop of Jmeter nodes. Control the Jmeter tool of the Jmeter node through the subnode manager module to execute the performance test script, generate test logs, and use the FileBeat of the Jmeter node to asynchronously parse the test logs, and submit the parsed log files to the Kafka message queue.

[0007] The submitted log files are forwarded to the report generation component through the Kafka message queue. The parameter fields in the log files are parsed by the report generation component, and a performance test report is generated according to the parameter fields. The parameter fields include: timestamp, time consumption, label, return code, return information, thread name, data type, success flag, failure information, number of bytes, sent message bytes, request address, delay, idle time and connection.

[0008] The generated performance test report is displayed through the display module.

[0009] Furthermore, the method for implementing a large-concurrency performance testing platform based on log file parsing uploads Jmeter performance testing scripts through a script manager module using a Web interface and stores them in a specified directory on the server, while also providing viewing, editing and deleting functions of the Jmeter performance testing scripts using the Web interface.

[0010] Furthermore, the method for implementing a large concurrent performance testing platform based on log file parsing distributes the files of the Jmeter performance testing script to each Jmeter node based on the Socket communication protocol through a sub-node manager module, and uses remote commands or API calls to control the start and stop of the Jmeter node to perform performance testing.

[0011] Furthermore, the performance test report generated by the report generation component of the method for implementing a large-concurrency performance test platform based on log file parsing includes charts, statistical data and abnormal information, and the performance test report is displayed by the display module using a Web interface or email.

[0012] The present invention also provides a large concurrent performance testing platform based on log file parsing, the large concurrent performance testing platform includes a script manager module, a subnode manager module, a report generation component and a display module.

[0013] The script manager module uploads, stores and manages Jmeter performance test scripts. The subnode manager module distributes the specified Jmeter performance test scripts to all Jmeter nodes and controls the start and stop of Jmeter nodes. The subnode manager module controls the Jmeter tool of the Jmeter node to execute the performance test script, generate test logs, and use the FileBeat of the Jmeter node to asynchronously parse the test logs and submit the parsed log files to the Kafka message queue.

[0014] The submitted log files are forwarded to the report generation component through the Kafka message queue. The report generation component parses the parameter fields in the log files and generates a performance test report based on the parameter fields. The parameter fields include: timestamp, time consumption, label, return code, return information, thread name, data type, success flag, failure information, number of bytes, sent message bytes, request address, delay, idle time and connection.

[0015] The display module displays the generated performance test report.

[0016] Furthermore, the script manager module of the large concurrent performance testing platform based on log file parsing uses a Web interface to upload Jmeter performance testing scripts and stores them in a designated directory on the server, while using the Web interface to provide viewing, editing and deletion functions of the Jmeter performance testing scripts.

[0017] Furthermore, the subnode manager module of the large concurrent performance testing platform based on log file parsing distributes the files of the Jmeter performance testing script to each Jmeter node based on the Socket communication protocol, and uses remote commands or API calls to control the start and stop of the Jmeter node to execute performance testing.

[0018] Furthermore, the performance test report generated by the report generation component of the large-concurrency performance test platform based on log file parsing includes charts, statistical data and abnormal information, and the display module uses a Web interface or email to display the performance test report.

[0019] The benefits of the present invention are:

[0020] Through the platform's implementation method, the resource requirements in large concurrent tests are significantly reduced, and the accuracy of performance tests is greatly improved. The platform uses efficient log parsing technology to accurately capture performance data and ensure that the test results are accurate.

[0021] The platform scheduling script makes large-scale concurrent testing more convenient and efficient than ever before, greatly shortening the test cycle and reducing resource and time costs. It also greatly reduces manpower investment and further compresses the overall cost of performance testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic diagram of the platform architecture of the present invention. DETAILED DESCRIPTION

[0023] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0024] Example 1

[0025] The present invention provides a method for implementing a large concurrent performance test platform based on log file analysis, and establishes a large concurrent performance test platform. The large concurrent performance test platform includes a script manager module, a subnode manager module, a report generation component and a display module.

[0026] Upload, store and manage Jmeter performance test scripts through the script manager module. The script manager module uses the web interface to upload Jmeter performance test scripts and stores them in the specified directory on the server. The web interface also provides the functions of viewing, editing and deleting Jmeter performance test scripts.

[0027] The subnode manager module distributes the specified Jmeter performance test script files to each Jmeter node based on the Socket communication protocol, and uses remote commands or API calls to control the start and stop of the Jmeter node to perform performance testing. The subnode manager module controls the Jmeter tool of the Jmeter node to execute the performance test script, generate test logs, and use the FileBeat of the Jmeter node to asynchronously parse the test logs, and submit the parsed log files to the Kafka message queue.

[0028] The Jmeter node installs Jmeter and FileBeat, configures FileBeat, specifies the path of the Jmeter log file, and configures Kafka as the output target.

[0029] The configuration code can be referred to as follows:

[0030]

[0031]

[0032] The submitted log files are forwarded to the report generation component through the Kafka message queue. The Kafka message queue saves and forwards all the data submitted by FileBeat. Kafka is installed and configured in the cluster, and a topic is created to receive the data submitted by FileBeat. Kafka persists the received data to the disk to ensure the reliability and recoverability of the data. Kafka forwards the received data to the report generation component for real-time processing.

[0033] The report generation component parses the parameter fields in the log file and generates a performance test report based on the parameter fields, where the parameter fields include: timestamp, time consumption, label, return code, return information, thread name, data type, success flag, failure information, number of bytes, sent message bytes, request address, delay, idle time and connection.

[0034] The generated performance test report is displayed through the display module.

[0035] The parameter fields can be found in Table 1.

[0036] Table 1

[0037] Parameter fields describe timeStamp Timestamp elapsed Time consumption label Label responseCode Return Code responseMessage Return information threadName Thread Name dataType Data Types success Success Sign failureMessage Failure Information bytes Number of bytes sentBytes Send message bytes grpThreads Threads allThreads All threads URL Request address Latency Delay IdleTime Free time Connect connect

[0038] The performance test report generated by the report generation component may include charts, statistical data and exception information, and the performance test report is displayed through the display module using the web interface or email.

[0039] The platform established by the method of the present invention has the following advantages:

[0040] High concurrent performance: Through distributed nodes and Kafka message queues, high concurrent performance testing and data transmission can be achieved.

[0041] Real-time: FileBeat asynchronously parses logs and submits them to Kafka in real time. The report generation component can generate performance test reports in real time.

[0042] Scalability: The platform architecture is easily scalable and performance can be improved by increasing the number of distributed nodes and Kafka cluster nodes.

[0043] Flexibility: Supports a variety of performance test scripts and log parsing methods, which can be customized and expanded according to actual needs.

[0044] Example 2

[0045] The present invention also provides a large concurrent performance testing platform based on log file parsing, the large concurrent performance testing platform includes a script manager module, a subnode manager module, a report generation component and a display module.

[0046] The script manager module uploads, stores and manages Jmeter performance test scripts. The subnode manager module distributes the specified Jmeter performance test scripts to all Jmeter nodes and controls the start and stop of Jmeter nodes. The subnode manager module controls the Jmeter tool of the Jmeter node to execute the performance test script, generate test logs, and use the FileBeat of the Jmeter node to asynchronously parse the test logs and submit the parsed log files to the Kafka message queue.

[0047] The submitted log files are forwarded to the report generation component through the Kafka message queue. The report generation component parses the parameter fields in the log files and generates a performance test report based on the parameter fields. The parameter fields include: timestamp, time consumption, label, return code, return information, thread name, data type, success flag, failure information, number of bytes, sent message bytes, request address, delay, idle time and connection.

[0048] The display module displays the generated performance test report.

[0049] As the information interaction and execution process between the modules in the above platform are based on the same concept as the embodiment of the method of the present invention, the specific contents can be found in the description of the embodiment of the method of the present invention and will not be repeated here.

[0050] Similarly, the platform of the present invention significantly reduces the resource requirements in large concurrent tests and greatly improves the accuracy of performance tests. The platform uses efficient log parsing technology to accurately capture performance data and ensure that the test results are accurate.

[0051] The platform scheduling script makes large-scale concurrent testing more convenient and efficient than ever before, greatly shortening the test cycle and reducing resource and time costs. It also greatly reduces manpower investment and further compresses the overall cost of performance testing.

[0052] It should be noted that not all steps and modules in the above-mentioned processes and platform structures are necessary, and some steps or modules can be ignored according to actual needs. The execution order of each step is not fixed and can be adjusted as needed. The system structure described in the above-mentioned embodiments can be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or some components in multiple independent devices may be implemented together.

[0053] The above-described embodiments are only preferred embodiments for fully illustrating the present invention, and the protection scope of the present invention is not limited thereto. Equivalent substitutions or changes made by those skilled in the art based on the present invention are within the protection scope of the present invention. The protection scope of the present invention shall be subject to the claims.

Claims

1. A method for implementing a large concurrent performance test platform based on log file parsing, characterized by: Establish a large concurrent performance testing platform, which includes a script manager module, a subnode manager module, a report generation component and a display module. Upload, store and manage Jmeter performance test scripts through the script manager module, distribute the specified Jmeter performance test scripts to all Jmeter nodes through the subnode manager module, and control the start and stop of Jmeter nodes. Control the Jmeter tool of the Jmeter node through the subnode manager module to execute the performance test script, generate test logs, and use the FileBeat of the Jmeter node to asynchronously parse the test logs, and submit the parsed log files to the Kafka message queue. The submitted log files are forwarded to the report generation component through the Kafka message queue. The parameter fields in the log files are parsed by the report generation component, and a performance test report is generated according to the parameter fields. The parameter fields include: timestamp, time consumption, label, return code, return information, thread name, data type, success flag, failure information, number of bytes, sent message bytes, request address, delay, idle time and connection. The generated performance test report is displayed through the display module.

2. According to the method for implementing a large concurrent performance test platform based on log file parsing according to claim 1, it is characterized by: The Jmeter performance test script is uploaded through the Web interface through the script manager module and stored in the specified directory on the server. At the same time, the Web interface provides the viewing, editing and deletion functions of the Jmeter performance test script.

3. According to the method for implementing a large concurrent performance test platform based on log file parsing according to claim 1, it is characterized by: The subnode manager module distributes the Jmeter performance test script files to each Jmeter node based on the Socket communication protocol, and uses remote commands or API calls to control the start and stop of the Jmeter node to perform performance testing.

4. The method for implementing a large concurrent performance test platform based on log file parsing according to claim 1 is characterized in that The performance test report generated by the report generation component includes charts, statistical data and exception information. The performance test report is displayed through the display module using the web interface or email.

5. A large concurrent performance testing platform based on log file parsing, characterized by The large concurrent performance testing platform includes a script manager module, a subnode manager module, a report generation component and a display module. The script manager module uploads, stores and manages Jmeter performance test scripts. The subnode manager module distributes the specified Jmeter performance test scripts to all Jmeter nodes and controls the start and stop of Jmeter nodes. The subnode manager module controls the Jmeter tool of the Jmeter node to execute the performance test script, generate test logs, and use the FileBeat of the Jmeter node to asynchronously parse the test logs and submit the parsed log files to the Kafka message queue. The submitted log files are forwarded to the report generation component through the Kafka message queue. The report generation component parses the parameter fields in the log files and generates a performance test report based on the parameter fields. The parameter fields include: timestamp, time consumption, label, return code, return information, thread name, data type, success flag, failure information, number of bytes, sent message bytes, request address, delay, idle time and connection. The display module displays the generated performance test report.

6. A large concurrent performance testing platform based on log file parsing according to claim 5, characterized in that The script manager module uses the Web interface to upload Jmeter performance test scripts and stores them in a specified directory on the server. It also uses the Web interface to provide viewing, editing, and deleting functions for Jmeter performance test scripts.

7. The large concurrent performance testing platform based on log file parsing according to claim 5 is characterized by: The subnode manager module distributes the Jmeter performance test script files to each Jmeter node based on the Socket communication protocol, and uses remote commands or API calls to control the start and stop of the Jmeter node to perform performance testing.

8. The large concurrent performance testing platform based on log file parsing according to claim 5 is characterized by: The performance test report generated by the report generation component includes charts, statistical data and exception information, and the display module uses a web interface or email to display the performance test report.

Images