Full-link stress testing method

Abstract

The present invention discloses a full-link stress testing method, including a stress testing preparation module, a request initiation module, an execution machine, a data acquisition module and a data analysis module; the following steps are followed: S1: the stress testing preparation module prepares stress testing data before stress testing, and cuts and generates new data according to the number of stress testing terminals; S2: the request initiation module initiates data download and stress testing requests to the execution machine, and the execution machine sends the stress testing request to the stress testing terminal; S3: when the stress testing terminal executes the stress testing request, the data acquisition module collects resources and calls data of the stress testing terminal; S4: the collected data is aggregated, and the data analysis module performs stress testing analysis based on the aggregated data. The present invention can quickly and accurately locate the service method and the corresponding code line with performance problems layer by layer from the request initiation terminal to the stress testing terminal, and the efficient data analysis method improves the stress testing efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of software system testing, and in particular to a full-link stress testing method. Background Art

[0002] Link stress testing is a process of stress testing the entire business chain based on actual production business scenarios and system environments, simulating massive user requests and data, and continuously optimizing the process. At present, business scenarios are becoming more and more complex. In the entire business chain, the correlation between different application services is becoming closer and closer, which can be said to be a domino effect. However, in the current relevant link stress testing solutions, basically only stress testing is initiated on the system. The data generated in each link is scattered in various systems. Performance analysis is usually carried out by stress testing personnel, operation and maintenance personnel, and R&D personnel to conduct correlation analysis to find the root cause of the problem. Stress testing and analysis are not integrated, resulting in high stress testing costs and progress. Summary of the invention

[0003] The purpose of the present invention is to provide a full-link stress testing method. The present invention can quickly and accurately locate the service method with performance problems and the corresponding code line layer by layer from the request initiator to the stress testing end, and the efficient data analysis method improves the stress testing efficiency.

[0004] In order to solve the above technical problems, the present invention provides the following technical solutions: a full-link stress testing method, including a stress testing preparation module, a request initiation module, an execution machine, a data acquisition module and a data analysis module; and is performed according to the following steps:

[0005] S1: The stress test preparation module prepares stress test data before stress testing and generates new data by cutting it according to the number of stress test terminals.

[0006] S2: The request initiation module initiates data download and stress testing requests to the execution machine, and the execution machine sends the stress testing request to the stress testing end;

[0007] S3: When the stress test request is executed on the stress test end, the data collection module is used to collect resources and call data on the stress test end;

[0008] S4: Aggregate the collected data, and the data analysis module performs stress testing analysis based on the aggregated data.

[0009] The above-mentioned full-link stress testing method, the stress testing analysis also includes link perception and JVM analysis, and provides layer-by-layer analysis capabilities from the request initiation module to the stress testing end when the stress testing end performs stress testing. The specific implementation steps are as follows:

[0010] A. Before performing stress testing, configure the expected values ​​of stress testing indicators according to actual business needs; the expected values ​​include stress testing end indicators and tested service resource indicators. Stress testing end indicators include average response time, number of failures, and number of successful requests per second; tested service resource indicators include CPU usage, memory usage, disk read and write speed, network read and write speed, and packet loss rate;

[0011] B. During the stress test, determine in real time whether the indicator data of each request exceeds the expected value of the configuration. If it exceeds, an alarm is triggered immediately, and the performance bottleneck point analysis is triggered based on the alarm;

[0012] C. The performance bottleneck analysis process is to first check the request point graph distribution in the last 5 minutes to find the links that are obviously not in line with expectations, check the time ranking of each node in the link, and trigger the corresponding service process analysis for the most time-consuming node.

[0013] The aforementioned full-link stress testing method uses alarms to determine whether there are performance problems with requests from the perspective of the stress testing end: the stress testing end uses a sliding window to query whether the average values ​​of the last 30 seconds all exceed the alarm value based on the current time. If so, an alarm is issued; the interval between two alarms needs to be greater than 10 minutes; the alarm method is that the alarm judgment service will store the alarm information in the third-party cache, and the alarm data will be pushed to the data analysis module in real time through the message middleware.

[0014] In the aforementioned full-link stress testing method, the alarm information includes the occurrence time, alarm object, alarm indicator, expected value, actual value and requested traceID;

[0015] The traceID is obtained by sending the alarm configuration to the execution machine before the stress test starts. The execution machine records a traceID per second for each request record that exceeds the threshold and passes it to the data analysis module.

[0016] In the aforementioned full-link stress testing method, in step S4, data aggregation includes request data aggregation and link data aggregation;

[0017] The request data aggregation adopts two layers of aggregation. The first layer is to aggregate the requests sent per second by the execution machine, and calculate the 90th, 95th and 99th percentiles of the request time during the aggregation; the second layer is to aggregate through a sliding window method with a window size of 3 seconds, and aggregate the data reported by different execution machines for the same request at the same time;

[0018] The link data aggregation includes single request link aggregation and request name aggregation;

[0019] The single request link aggregation is to automatically generate a globally unique traceID at the data collection point of the first service under test in the request link, and the traceID is passed to each service under test link collection point along the call flow. The link collection point automatically carries the traceID when reporting the collection data, and a complete request link is aggregated in chronological order according to the traceID;

[0020] The request name aggregation is that when the execution machine sends a request, it will automatically put the request name in the request header, and the data collection record of the first tested service in the request link will report the request name to the aggregation module, and the aggregation module will aggregate the link data per second in real time according to the request name.

[0021] In the aforementioned full-link stress testing method, the aggregated data includes request data and request error logs.

[0022] The request data includes request name, total number of successful requests, total number of failed requests, average response time, request response status code, total request response time and request time percentile data;

[0023] The request error log includes the request name, failure number, failure summary, failure details and keywords.

[0024] In the aforementioned full-link stress testing method, a link distribution point diagram is generated for each request in the data analysis module;

[0025] In the link distribution point diagram, link information includes request duration, associated traceID, and time. The page is displayed as a scatter plot with the x-axis representing time and the y-axis representing request duration. The points on the diagram show the corresponding link details.

[0026] In the aforementioned full-link stress testing method, each node information in the link details includes the reporting end ID and the class to which the method belongs. A trigger analysis request is sent to the corresponding reporting end through a remote call, and the reporting end reports the data in real time through messages.

[0027] The aforementioned full-link stress testing method reports data including method CPU usage ranking, CPU load analysis, deadlock analysis, thread pool creation and destruction analysis, decompilation of specified methods, process memory recovery and occupancy analysis. The recovery analysis includes the number of recovery times, time consumption and trend analysis, and the occupancy analysis includes the occupancy ratio and size of the old generation, young generation and permanent generation.

[0028] In the aforementioned full-link stress testing method, each point in the link distribution point diagram represents a call link. The call link display dimensions include call topology diagram, call tree, and call analysis.

[0029] The call topology diagram describes the call vector diagram between each node, the health status of each node, and the number of calls between two nodes;

[0030] The call tree includes the call node, the actual time consumption of the method of each node, the request data and the response data of the method;

[0031] The call analysis includes sorting out the third-party services, messages, databases, caches, and method time-consuming rankings involved in the link call.

[0032] Compared with the prior art, the present invention initiates data download and stress testing requests to the execution machine through the request initiation module, and the execution machine sends the stress testing request to the stress testing end. When the stress testing end executes the stress testing request, the data collection module collects the resources and call data of the stress testing end; the collected data is aggregated, and the data analysis module performs stress testing analysis based on the aggregated data. The present invention can quickly and accurately locate the service method and the corresponding code line with performance problems layer by layer from the request initiation end to the stress testing end, and the efficient data analysis method improves the efficiency of stress testing. In addition, the present invention completes performance stress testing, link perception and jvm analysis on the same stress testing end, breaking through the barriers between performance stress testing and link analysis and jvm analysis, and providing layer-by-layer analysis capabilities from the stress testing end to the application under test. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a schematic diagram of the process of the present invention

[0034] Figure 2 This is a schematic diagram of the performance bottleneck analysis process. DETAILED DESCRIPTION

[0035] The present invention is further described below in conjunction with the embodiments and drawings, but they are not intended to limit the present invention.

[0036] Embodiment: A full-link stress testing method includes a stress testing preparation module, a request initiation module, an execution machine, a data acquisition module, and a data analysis module; and is performed according to the following steps:

[0037] S1: The stress test preparation module prepares stress test data before stress testing and generates new data by cutting it according to the number of stress test terminals (i.e., stress test book). After cutting, the stress test data corresponds to the stress test terminals one by one.

[0038] S2: The request initiation module initiates data download and stress testing requests to the execution machine, and the execution machine sends the stress testing request to the stress testing end;

[0039] S3: When the stress testing request is executed on the stress testing end, data collection on the stress testing end is performed through the data collection module; the data collection module adopts bytecode enhancement technology to automatically intercept and enhance the code of the stress testing end without modifying the code of the stress testing end and without affecting the business logic of the stress testing end, so as to realize the link tracking function, that is, through the JavaAgent mechanism, the bytecode of the stress testing end is dynamically modified before the class is loaded, and the data collection logic is added before and after the original business logic, so as to realize the dynamic enhancement of the capability of the stress testing end without modifying the source code of the stress testing end, and realize the link tracking function and the execution code collection function.

[0040] S4: Aggregate the collected data, and the data analysis module performs stress testing analysis based on the aggregated data.

[0041] In this embodiment, data aggregation includes request data aggregation and link data aggregation;

[0042] The request data aggregation adopts two layers of aggregation. The first layer is to aggregate the requests sent per second by the execution machine, and calculate the 90th, 95th and 99th percentiles of the request time during the aggregation; the second layer is to aggregate through a sliding window method with a window size of 3 seconds, and aggregate the data reported by different execution machines for the same request at the same time;

[0043] The link data aggregation includes single request link aggregation and request name aggregation;

[0044] The single request link aggregation is to automatically generate a globally unique traceID at the data collection point of the first service under test in the request link, and the traceID is passed to each service under test link collection point along the call flow. The link collection point automatically carries the traceID when reporting the collection data, and a complete request link is aggregated in chronological order according to the traceID;

[0045] The request name aggregation is that when the execution machine sends a request, it will automatically put the request name in the request header, and the data collection record of the first tested service in the request link will report the request name to the aggregation module, and the aggregation module will aggregate the link data per second in real time according to the request name.

[0046] Aggregated data includes request data and request error logs;

[0047] The request data includes request name, total number of successful requests, total number of failed requests, average response time, request response status code, total request response time and request time percentile data;

[0048] The request error log includes the request name, failure number, failure summary, failure details and keywords.

[0049] In this embodiment, the stress test analysis also includes link perception and JVM analysis, and provides layer-by-layer analysis capabilities from the request initiation module to the stress test end when the stress test end performs stress testing. The specific implementation steps are as follows:

[0050] A. Before performing stress testing, configure the expected values ​​of stress testing indicators according to actual business needs; the expected values ​​include stress testing end indicators and tested service resource indicators. Stress testing end indicators include average response time, number of failures, and number of successful requests per second; tested service resource indicators include CPU usage, memory usage, disk read and write speed, network read and write speed, and packet loss rate;

[0051] B. During the stress test, determine in real time whether the indicator data of each request exceeds the expected value of the configuration. If it exceeds, an alarm is triggered immediately, and the performance bottleneck point analysis is triggered based on the alarm;

[0052] Use stress test alarms to determine whether there are performance issues with requests from the stress test end:

[0053] Alarm on the stress test end:

[0054] Alarms are divided into global alarms and scenario alarms. Global alarms are effective for all scenarios, while scenario alarms are only effective for the currently running scenario. Scenario alarms override global alarms. For example, if the global alarm requires the CPU usage rate not to exceed 80%, but the scenario requires the CPU usage rate not to exceed 60%, then 60% is used to determine whether the current CPU usage rate meets expectations.

[0055] The stress test end alarm uses a sliding window method to query whether the average values ​​of the last 30 seconds all exceed the alarm value based on the current time. If so, an alarm is issued. The interval between two alarms needs to be greater than 10 minutes.

[0056] The alarm method is to store the alarm information in the third-party cache through the alarm judgment service, use the web service to read the cache in real time and push the alarm data to the data analysis module in real time through the message middleware.

[0057] The alarm information includes the occurrence time, alarm object, alarm indicator, expected value, actual value and requested traceID;

[0058] The traceID is obtained by sending the alarm configuration to the execution machine before the stress test starts. The execution machine records a traceID per second for each request record that exceeds the threshold and passes it to the data analysis module.

[0059] C. Figure 2 As shown in the figure, the performance bottleneck analysis process is:

[0060] Step 201: First, check the link request point graph in the last 5 minutes, and judge in real time whether the request duration meets expectations, thereby exposing requests with performance issues;

[0061] Step 202: Find the link that obviously does not meet expectations, that is, find the slowest link in the last 5 minutes;

[0062] Step 203: Check the time ranking of each node in the link and analyze the slowest node method in the link;

[0063] Step 204: triggering the corresponding service process analysis for the most time-consuming node. In this embodiment, the Javaagent is triggered to perform CPU hotspot, thread and memory analysis.

[0064] Among them, the link request point graph is generated for each request in the data analysis module;

[0065] In the link distribution point diagram, link information includes request duration, associated traceID, and time. The page is displayed as a scatter plot, with the x-axis representing time and the y-axis representing request duration. The points on the diagram show the corresponding link details. The call link display dimensions include call topology diagram, call tree, and call analysis.

[0066] The call topology diagram describes the call vector diagram between each node, the health status of each node, and the number of calls between two nodes;

[0067] The call tree includes the call node, the actual time consumption of the method of each node, the request data and the response data of the method;

[0068] The call analysis includes sorting out the third-party services, messages, databases, caches, and method time-consuming rankings involved in the link call.

[0069] The information of each node in the link details includes the reporting end ID and the class to which the method belongs. A trigger analysis request is sent to the corresponding reporting end through a remote call. The reporting end reports the data in real time through messages. The reported data includes the CPU usage ranking of the method, CPU load analysis, deadlock analysis, thread pool creation and destruction analysis, decompilation of the specified method, process memory recovery and occupancy analysis. The recovery analysis includes the number of recovery times, time consumption and trend analysis, and the occupancy analysis includes the occupancy ratio and size of the old generation, young generation and permanent generation.

[0070] In summary, the present invention can quickly and accurately locate the service method and the corresponding code line with performance problems layer by layer from the request initiator to the stress testing end, and the efficient data analysis method improves the stress testing efficiency.

Claims

1. A full-link stress testing method, including a stress testing preparation module, a request initiation module, an execution machine, a data acquisition module and a data analysis module; the characteristics are: Follow these steps: S1: The stress test preparation module prepares stress test data before stress testing and generates new data by cutting it according to the number of stress test terminals. S2: The request initiation module initiates data download and stress testing requests to the execution machine, and the execution machine sends the stress testing request to the stress testing end; S3: When the stress test request is executed on the stress test end, the data collection module is used to collect resources and call data on the stress test end; S4: Aggregate the collected data, and the data analysis module performs stress testing analysis based on the aggregated data; The stress test analysis includes link perception and JVM analysis, and provides layer-by-layer analysis capabilities from the request initiation module to the stress test end when the stress test is performed on the stress test end. The specific implementation steps are as follows: A. Before performing stress testing, configure the expected values ​​of stress testing indicators according to actual business needs; the expected values ​​of stress testing indicators include stress testing end indicators and tested service resource indicators. Stress testing end indicators include average response time, number of failures, and number of successful requests per second; tested service resource indicators include CPU usage, memory usage, disk read and write speed, network read and write speed, and packet loss rate; B. During the stress test, determine in real time whether the indicator data of each request exceeds the expected value of the configuration. If it exceeds, an alarm is triggered immediately, and the performance bottleneck point analysis is triggered based on the alarm; C. The performance bottleneck analysis process is to first check the request point graph of the last 5 minutes to find out the links that are obviously not in line with expectations, check the time ranking of each node in the link, and trigger the corresponding service process analysis for the most time-consuming node; In step S4, data aggregation includes request data aggregation and link data aggregation; The request data aggregation adopts two-layer aggregation. The first layer is to aggregate the requests sent per second by the execution machine, and calculate the 90th, 95th and 99th percentiles of the request time during aggregation; the second layer is to aggregate through a sliding window method with a window size of 3 seconds, and aggregate the data reported by different execution machines for the same request at the same time; The link data aggregation includes single request link aggregation and request name aggregation; The single request link aggregation is to automatically generate a globally unique traceID at the data collection point of the first service under test in the request link, and the traceID is passed to each service under test link collection point along the call flow. The link collection point automatically carries the traceID when reporting the collection data, and a complete request link is aggregated in chronological order according to the traceID; The request name aggregation is that the execution machine automatically puts the request name into the request header when sending a request, and the data collection record of the first tested service of the request link reports the request name to the aggregation module, and the aggregation module aggregates the link data per second in real time according to the request name.

2. The full-link stress testing method according to claim 1 is characterized in that: Use alarms to determine whether there are performance problems with requests from the stress test end: Alarms on the stress test end use a sliding window to query whether the average values ​​of the last 30 seconds all exceed the alarm value based on the current time. If so, an alarm is issued. The interval between two alarms must be greater than 10 minutes. The alarm method is to push the alarm information to the data analysis module in real time through the alarm judgment service via the message middleware.

3. The full-link stress testing method according to claim 2 is characterized in that: The alarm information includes occurrence time, alarm object, alarm indicator, expected value, actual value and requested traceID; The traceID is obtained by sending the alarm configuration to the execution machine before the stress test begins. The execution machine records a traceID per second for each request record that exceeds the threshold and passes it to the data analysis module.

4. The full-link stress testing method according to claim 1 is characterized in that: Aggregated data includes request data and request error logs; The request data includes request name, total number of successful requests, total number of failed requests, average response time, request response status code, total request response time and request time percentile data; The request error log includes the request name, failure number, failure summary, failure details and keywords.

5. The full-link stress testing method according to claim 1 is characterized in that: Generate a link distribution point graph for each request in the data analysis module; In the link distribution point diagram, link information includes request duration, associated traceID, and time. The page is displayed as a scatter plot with the x-axis representing time and the y-axis representing request duration. The points on the diagram show the corresponding link details.

6. The full-link stress testing method according to claim 5 is characterized in that: The information of each node in the link details includes the reporting end ID and the class where the method is located. A trigger analysis request is sent to the corresponding reporting end through a remote call, and the reporting end reports the data in real time through messages.

7. The full-link stress testing method according to claim 6 is characterized in that: The reported data includes the CPU usage ranking of methods, CPU load analysis, deadlock analysis, thread pool creation and destruction analysis, decompilation of code for specified methods, process memory recovery and occupancy analysis. The recovery analysis includes the number of recovery times, time consumption and trend analysis, and the occupancy analysis includes the occupancy ratio and size of the old generation, young generation and permanent generation.

8. The full-link stress testing method according to claim 5, characterized in that: In the link distribution point diagram, each point on the diagram represents a call link. The call link display dimensions include call topology diagram, call tree and call analysis. The call topology diagram describes the call vector diagram between each node, the health status of each node, and the number of calls between two nodes; The call tree includes the call node, the actual time consumption of the method of each node, the request data and the response data of the method; The call analysis includes sorting out the third-party services, messages, databases, caches, and method time-consuming rankings involved in the link call.

Images