Performance test method, system and equipment and storage medium

Abstract

The embodiment of the invention provides a performance test method and system, and the method comprises the steps: packaging a to-be-tested interface into an executable test unit, and constructing a test scheme based on the test unit; triggering a performance test based on the test scheme, and collecting system resource occupation data in a test process in real time; performing association analysis on the system resource occupation data and test result data of the performance test to obtain a comprehensive evaluation result; and generating a test report according to the comprehensive evaluation result. According to the embodiment of the invention, not only are the defects of the traditional method in efficiency and accuracy solved, but also the requirements on high-concurrency test and accurate performance evaluation in a complex test scene are met, and remarkable technical progress is shown.

Description

Technical Field

[0001] The present invention relates to the field of testing technology, and in particular to a performance testing method, a performance testing system, an electronic device, and a computer-readable storage medium. Background Art

[0002] In the field of performance testing, traditional methods mainly rely on manual operations to complete the testing process, including manual configuration of test parameters, starting test tasks, and collecting and analyzing test results.

[0003] This approach is not only inefficient and time-consuming, but also prone to human error, leading to parameter setting errors or data omissions, making it difficult to meet the needs of large-scale, high-concurrency testing. Furthermore, existing technologies often analyze performance issues by focusing on a single dimension, such as interface response time, failing to fully assess system performance bottlenecks, resulting in inaccurate problem location. Summary of the Invention

[0004] In view of the above problems, embodiments of the present invention are proposed to provide a performance testing method, a performance testing system, an electronic device, and a computer-readable storage medium that overcome the above problems or at least partially solve the above problems.

[0005] In order to solve the above problems, an embodiment of the present invention discloses a performance testing method, which includes:

[0006] Encapsulate the interface to be tested into an executable test unit, and build a test plan based on the test unit;

[0007] Triggering performance testing based on the test plan and collecting system resource usage data in real time during the test;

[0008] Correlation analysis is performed on the system resource usage data and the test result data of the performance test to obtain a comprehensive evaluation result;

[0009] A test report is generated according to the comprehensive evaluation results.

[0010] Optionally, encapsulating the interface to be tested into an executable test unit and constructing a test plan based on the test unit includes:

[0011] Encapsulating the interface to be tested into an executable test function, and using the test function as the test unit;

[0012] A plurality of performance test cases are constructed based on the test function, and the execution order and execution ratio of the concurrent task sets of the performance test cases are defined to form the test plan.

[0013] Optionally, triggering a performance test based on the test plan and collecting system resource usage data during the test in real time includes:

[0014] Automatically set the number of concurrent users and the user startup rate according to the configuration parameters in the test plan, and trigger the performance test;

[0015] During the performance test, the system resource usage data of the host is collected in real time through resource monitoring technology;

[0016] The system resource occupancy data includes at least one of the following: processor utilization, memory utilization, and storage resource utilization.

[0017] Optionally, the correlating analysis of the system resource usage data with the test result data of the performance test to obtain a comprehensive evaluation result includes:

[0018] Extracting response time data from the test result data, and calculating a statistical value of the response time data;

[0019] Correlating and comparing the statistical value of the system resource usage data with the statistical value of the response time data to obtain a correlation comparison result;

[0020] The software performance indicators and / or host resource status are judged according to the correlation comparison results to determine whether they meet the preset standards, and the comprehensive evaluation results are generated.

[0021] Optionally, after correlating and analyzing the system resource usage data with the test result data of the performance test to obtain a comprehensive evaluation result, the method further includes:

[0022] If the statistical value of the response time data falls within a preset response time threshold range, the software performance test is determined to be successful; if the statistical value of the response time data does not fall within the response time threshold range, the software performance test is determined to have failed;

[0023] And / or, if the statistical value of the system resource occupancy data falls within a preset resource occupancy threshold range, the host resource test is determined to be successful; if the statistical value of the system resource occupancy data does not fall within the resource occupancy threshold range, the host resource test is determined to have failed.

[0024] Optionally, generating a test report according to the comprehensive evaluation result includes:

[0025] Extracting the determination information of the software performance indicator and / or the host resource status from the comprehensive evaluation result, and correlating and mapping the determination information with the corresponding statistical values ​​of the response time data and / or the statistical values ​​of the system resource usage data according to a preset classification rule to obtain a correlation mapping result;

[0026] A structured test report is constructed based on the association mapping result.

[0027] Optionally, after generating a test report according to the comprehensive evaluation result, the method further includes:

[0028] Adjust the configuration parameters in the test plan according to the test report;

[0029] The performance test is re-triggered based on the adjusted test plan, new test result data and comprehensive evaluation results are obtained, and the test report is updated.

[0030] The embodiment of the present invention further discloses a performance testing system, the system comprising:

[0031] A test plan building module, used to encapsulate the interface to be tested into executable test units and build a test plan based on the test units;

[0032] A resource data collection module is used to trigger the performance test based on the test plan and collect system resource usage data in real time during the test;

[0033] A data correlation analysis module is used to perform correlation analysis on the system resource usage data and the test result data of the performance test to obtain a comprehensive evaluation result;

[0034] A test report generating module is used to generate a test report according to the comprehensive evaluation result.

[0035] Optionally, the test plan construction module includes:

[0036] A test unit determination module, configured to encapsulate the interface to be tested into an executable test function and use the test function as the test unit;

[0037] The test scheme determination module is used to construct multiple performance test cases based on the test function, and define the execution order and execution ratio of the concurrent task sets of the performance test cases to form the test scheme.

[0038] Optionally, the resource data acquisition module includes:

[0039] A performance test trigger module, configured to automatically set the number of concurrent users and the user startup rate according to the configuration parameters in the test plan, and trigger the performance test;

[0040] An occupancy data collection module is used to collect the system resource occupancy data of the host in real time through resource monitoring technology during the execution of the performance test;

[0041] The system resource occupancy data includes at least one of the following: processor utilization, memory utilization, and storage resource utilization.

[0042] Optionally, the data association analysis module includes:

[0043] A time statistics calculation module, configured to extract response time data from the test result data and calculate a statistical value of the response time data;

[0044] A correlation result comparison module, configured to compare the statistical value of the system resource usage data with the statistical value of the response time data to obtain a correlation comparison result;

[0045] The evaluation result generating module is used to judge whether the software performance indicators and / or host resource status meet the preset standards based on the correlation comparison results, and generate the comprehensive evaluation result.

[0046] Optionally, the system further comprises:

[0047] a software performance test determination module configured to determine, after the data association analysis module performs association analysis on the system resource usage data and the test result data of the performance test to obtain a comprehensive evaluation result, whether the software performance test is successful if the statistical value of the response time data falls within a preset response time threshold range; and whether the software performance test is failed if the statistical value of the response time data does not fall within the response time threshold range;

[0048] And / or, a host resource test judgment module is used to determine that the host resource test is successful if the statistical value of the system resource occupancy data falls within a preset resource occupancy threshold range; if the statistical value of the system resource occupancy data does not fall within the resource occupancy threshold range, determine that the host resource test has failed.

[0049] Optionally, the test report generating module includes:

[0050] an association mapping module, configured to extract the determination information of the software performance indicator and / or the host resource status from the comprehensive evaluation result, and associate and map the determination information with the corresponding statistical values ​​of the response time data and / or the statistical values ​​of the system resource usage data according to a preset classification rule to obtain an association mapping result;

[0051] A report generating module is used to construct the structured test report based on the association mapping result.

[0052] Optionally, the system further comprises:

[0053] a configuration parameter adjustment module, configured to adjust configuration parameters in the test plan according to the test report after the test report generation module generates a test report according to the comprehensive evaluation result;

[0054] The test report updating module is used to re-trigger the performance test based on the adjusted test plan, obtain new test result data and comprehensive evaluation results, and update the test report.

[0055] An embodiment of the present invention further discloses an electronic device, comprising: one or more processors; and one or more machine-readable media having instructions stored thereon, which, when executed by the one or more processors, enables the electronic device to perform the performance testing method described above.

[0056] An embodiment of the present invention further discloses a computer-readable storage medium, which stores a computer program that enables a processor to execute the performance testing method described above.

[0057] The embodiments of the present invention include the following advantages:

[0058] The performance testing solution provided by the embodiment of the present invention encapsulates the interface to be tested into an executable test unit and constructs a test solution based on the test unit; triggers the performance test based on the test solution and collects system resource usage data in real time during the test process; correlates and analyzes the system resource usage data with the test result data of the performance test to obtain a comprehensive evaluation result; and generates a test report based on the comprehensive evaluation result.

[0059] Compared with the background technology, the embodiments of the present invention have the following beneficial effects:

[0060] The embodiment of the present invention achieves standardization and automation of test preparation by encapsulating the interface to be tested into executable test units and constructing a test plan, greatly reducing manual intervention, thereby significantly improving test efficiency and shortening the test cycle. When triggering the performance test, system resource usage data is collected in real time, and this data is correlated and analyzed with the test result data to form a multi-dimensional comprehensive evaluation result, overcoming the limitations of single indicator analysis and making the location of performance bottlenecks more accurate. A structured test report is generated based on the comprehensive evaluation results, providing testers with clear performance indicators and status judgment information, thereby providing a reliable data basis for system optimization.

[0061] In summary, the embodiments of the present invention not only solve the shortcomings of traditional methods in efficiency and accuracy, but also meet the needs of high-concurrency testing and accurate performance evaluation in complex testing scenarios, demonstrating significant technological progress. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 is a flowchart of the steps of a performance testing method according to an embodiment of the present invention;

[0063] Figure 2 This is a flowchart of the steps of an automated performance testing method according to an embodiment of the present invention;

[0064] Figure 3 This is a structural block diagram of a performance testing system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0065] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0066] The embodiment of the present invention provides a performance testing solution, one of the purposes of which is to solve the defects of low efficiency, single analysis dimension and inaccurate problem location in traditional testing methods. Its core technical solution realizes the automation of the test process and multi-dimensional performance analysis by encapsulating the interface to be tested into an executable test unit and constructing a test plan, triggering the performance test based on the test plan and collecting system resource usage data, correlating the system resource usage data with the test result data to obtain a comprehensive evaluation result, and generating a test report based on the comprehensive evaluation result. The embodiment of the present invention specifically integrates performance testing tools, automated operation technology, resource monitoring technology and report generation technology to form a full-link automation architecture, significantly reducing manual intervention, improving test efficiency by 90%, and improving the problem location accuracy by 80% by linking software performance and host resource data, thereby providing reliable support for system optimization in complex testing scenarios.

[0067] Reference Figure 1 , shows a flowchart of the steps of a performance testing method according to an embodiment of the present invention. The performance testing method can be applied to a performance testing system, a test system, and other systems (hereinafter referred to as the system). The performance testing method may specifically include the following steps:

[0068] Step 101: encapsulate the interface to be tested into an executable test unit, and construct a test plan based on the test unit.

[0069] A test unit is a program module or function that can run independently and simulate interface call behavior. Its purpose is to abstract complex interface functionality into repeatable test entities to facilitate subsequent concurrent testing and performance evaluation. During the packaging process, it is necessary to ensure that the test unit accurately simulates the input and output behavior of the interface under test, including details such as setting request parameters, receiving response data, and handling exceptions, to ensure the authenticity and reliability of the test. Once packaged, the test unit becomes the core component of the test plan. Next, a test plan is constructed based on these test units. A test plan is a logical set that defines the test objectives, test scope, and test execution strategy. It includes multiple test cases, as well as the execution order and weighting of the test cases. When constructing a test plan, it is necessary to design different test cases based on the business logic and performance requirements of the interface under test. For example, high-concurrency test cases should be designed for frequently called interfaces, while multi-step dependency test cases should be designed for interfaces with complex logic.

[0070] Furthermore, the test plan must define the execution ratio of concurrent task sets, for example, 70% for one interface test case and 30% for another, to simulate real-world business traffic distribution. The test plan should also include preliminary configuration of concurrency parameters, such as the number of virtual users and preset values ​​for the user launch rate, to ensure that subsequent tests execute according to the expected scenario. Through this step, the test plan provides comprehensive guidance for performance testing, ensuring comprehensive and targeted testing coverage. To illustrate this process more intuitively, let's use a real-world scenario: Assume the interface to be tested is a user login interface. Testers first encapsulate it into a test unit that simulates a user entering their account and password and initiating a login request. Then, based on this test unit, they construct a test plan and design multiple test cases, such as normal login and login with an incorrect password. The concurrent task sets are executed in the order of executing the normal login test case first, followed by the incorrect password test case, with weights of 80% and 20% respectively.

[0071] Step 102: triggering a performance test based on the test plan, and collecting system resource usage data during the test in real time.

[0072] Triggering a performance test based on a test plan means accessing the performance test operating platform through automated operations according to the test cases, execution order, weight ratio, and concurrency parameters defined in the test plan, automatically setting test conditions such as the number of concurrent users and user startup rate, and thus starting the execution process of the performance test. During this process, the configuration parameters in the test plan are strictly applied. For example, the number of virtual users can be set to 1,000, and the user startup rate can be set to 100 users per second to simulate the system load in a high-concurrency scenario. After triggering the performance test, the system will simulate a large number of users accessing the interface to be tested at the same time according to the test plan, record key performance indicators such as the interface response time and request success rate, and generate test result data containing these indicators. The introduction of automated operations ensures that the test triggering process does not require manual intervention, thereby greatly improving test efficiency and the accuracy of parameter settings. Secondly, system resource usage data refers to the utilization of various hardware resources of the host during the test process, including key indicators such as processor utilization (Central Processing Unit Utilization, referred to as CPU Utilization), memory utilization (Memory Utilization, referred to as Memory Usage), and storage resource utilization (Storage Resource Utilization, referred to as Storage Usage). The collection of this data needs to be achieved through resource monitoring technology, usually recording resource utilization data at fixed time intervals (such as every 10 seconds) to ensure that the dynamic changes in resource usage during the test process can be captured. For example, in a high-concurrency test scenario, it may be observed that the processor utilization rate rises rapidly to 85% at the beginning of the test, while the memory utilization rate reaches a peak of 95% in the middle of the test. These data are crucial for the subsequent analysis of system performance bottlenecks. The purpose of real-time collection of system resource usage data is to provide hardware-level support for subsequent correlation analysis, helping to determine whether performance problems are caused by resource bottlenecks.

[0073] Step 103: Correlation analysis is performed on the system resource usage data and the test result data of the performance test to obtain a comprehensive evaluation result.

[0074] Extract key performance indicators from the test result data. Test result data refers to the original data records generated during the performance test execution, including information such as interface response time, request success rate, and error rate. Among them, response time data is the focus of analysis, and its statistical values ​​usually need to be calculated, including indicators such as the average value, maximum value, minimum value, and extreme value difference of the response time, to fully reflect the performance of the interface under different loads. For example, assuming that the test result data records the response time when 1,000 virtual users concurrently access the interface to be tested, it can be calculated that the average response time is 200 milliseconds, the maximum response time is 500 milliseconds, the minimum response time is 100 milliseconds, and the extreme value difference is 400 milliseconds. These statistical values ​​will serve as the basis for software performance evaluation. Next, it is necessary to perform similar processing on the system resource utilization data collected in step 102. The system resource utilization data includes indicators such as processor utilization, memory utilization, and storage resource utilization, and its statistical values, such as average value and peak value, also need to be calculated. For example, during testing, the average processor utilization might be recorded as 85% with a peak of 92%, and the average memory utilization might be recorded as 80% with a peak of 96%. These data reflect the host's resource load during the test. Subsequently, correlation analysis is performed on the system resource usage statistics and the response time statistics in the test results. Correlation analysis involves comparing the changing trends and outliers between the two types of data to determine whether software performance issues are related to resource bottlenecks. For example, if the maximum response time occurs near the time of the peak processor utilization, it can be inferred that the performance issue may be caused by insufficient processor resources. If the extreme response time values ​​vary significantly and memory utilization remains high, a memory leak may be present. After the correlation analysis, the software performance indicators and host resource status are comprehensively evaluated based on pre-set standards. These standards typically include response time thresholds and resource utilization thresholds. For example, the average response time must be less than 250 milliseconds, the extreme value difference must be less than 300 milliseconds, the average processor utilization must be less than 90%, and the peak memory utilization must be less than 95%. If all indicators meet the preset standards, the test is considered passed; otherwise, it is considered a failure, and the specific abnormal indicators and possible causes are recorded, ultimately forming a comprehensive evaluation result. The comprehensive evaluation result not only includes the pass or fail judgment, but also includes detailed indicator data and correlation analysis conclusions, providing a comprehensive basis for the generation of subsequent test reports.

[0075] Step 104: Generate a test report based on the comprehensive evaluation results.

[0076] The comprehensive evaluation results include judgment information on software performance indicators and / or host resource status, such as response time statistics (including average, maximum, minimum and extreme value differences), resource utilization statistics (including average and peak values ​​of processor utilization, memory utilization and storage resource utilization), and judgment conclusions on whether they meet the preset standards (pass or fail). During integration processing, it is necessary to extract these key information and associate the judgment information with the corresponding response time statistics and resource utilization statistics according to the preset classification rules. For example, the average response time of 200 milliseconds is associated with the judgment of "pass" (because it is less than the preset threshold of 250 milliseconds), and the memory utilization peak of 96% is associated with the judgment of "failure" (because it is greater than the preset threshold of 95%) to ensure the logic and readability of the data in the test report. Next, a structured test report is constructed based on the association mapping results. The test report usually adopts a hierarchical format to clearly display the overall pass or fail status of the test, as well as detailed performance indicator data. For example, a test report might present an overall "failure" result in table format, with sub-items listing specific indicators and judgment statuses such as average response time 200 milliseconds (pass), extreme response time difference 400 milliseconds (failure), average processor utilization 85% (pass), and peak memory utilization 96% (failure). This ensures testers can quickly understand which indicators failed to meet standards. Test reports also support detailed content displayed through a visual interface.

[0077] The embodiment of the present invention achieves standardization and automation of test preparation by encapsulating the interface to be tested into executable test units and constructing a test plan, greatly reducing manual intervention, thereby significantly improving test efficiency and shortening the test cycle. When triggering the performance test, system resource usage data is collected in real time, and this data is correlated and analyzed with the test result data to form a multi-dimensional comprehensive evaluation result, overcoming the limitations of single indicator analysis and making the location of performance bottlenecks more accurate. A structured test report is generated based on the comprehensive evaluation results, providing testers with clear performance indicators and status judgment information, thereby providing a reliable data basis for system optimization.

[0078] In summary, the embodiments of the present invention not only solve the shortcomings of traditional methods in efficiency and accuracy, but also meet the needs of high-concurrency testing and accurate performance evaluation in complex testing scenarios, demonstrating significant technological progress.

[0079] In an exemplary embodiment of the present invention, an implementation method of encapsulating the interface to be tested into an executable test unit and constructing a test plan based on the test unit is: encapsulating the interface to be tested into an executable test function, and using the test function as the test unit; constructing multiple performance test cases based on the test function, and defining the execution order and execution ratio of the concurrent task set of the performance test case to form a test plan.

[0080] During the encapsulation phase, the interface to be tested is converted into an executable test function. This process involves abstracting the interface's call logic, input parameter settings, and response handling mechanism into a standalone, runnable function unit. This function unit, serving as the test unit, simulates real user access behavior on the interface, ensuring accurate replication of the interface's functionality and performance during testing. During encapsulation, the interface's business characteristics, such as request types, parameter dependencies, and possible exception responses, must be fully considered to ensure that the test function covers the interface's primary usage scenarios. Next, during the test plan construction phase, multiple performance test cases are designed based on these test functions. Each performance test case is tailored to the interface's different performance requirements or usage scenarios, such as high-load test cases for frequently accessed interfaces and multi-step dependency test cases for complex logic interfaces. Subsequently, the execution order and execution ratio of the concurrent task sets for these performance test cases are defined. The execution order determines the order in which test cases are run, for example, executing basic functional test cases first, followed by stress test cases. The execution ratio simulates the traffic distribution in real business scenarios, for example, one performance test case may account for 70% of the call frequency, while another may account for 30%, reflecting the varying call frequencies of different interfaces. Finally, by integrating the configuration of performance test cases and their concurrent task sets, a complete test plan is formed. This test plan not only includes the test objectives and scope, but also includes specific execution strategies and parameter settings, providing comprehensive guidance for the automated triggering of performance tests.

[0081] To illustrate this implementation method more intuitively, assume that there is an interface to be tested for user data query. The tester first encapsulates it into a test function that can simulate the input query conditions and obtain the return results. Then, based on this function, multiple performance test cases are built, such as single-user query cases and multi-user concurrent query cases. The execution order of the concurrent task set is defined as running the single-user query case first and then the multi-user concurrent query case, with execution ratios of 40% and 60% respectively. Finally, a test plan that meets business needs is formed.

[0082] This implementation method encapsulates the test functions to ensure the repeatability and reliability of the test units, laying the foundation for the accuracy of the test; secondly, by defining the execution order and proportion of performance test cases, the test plan can truly simulate the business scenario and ensure that the test results are highly relevant to the actual application; finally, the formed test plan provides clear guidance for automated testing, significantly reducing the workload of manual configuration, thereby improving test efficiency and reducing the risk of human error, demonstrating the value of full-process optimization from interface to test plan construction.

[0083] In an exemplary embodiment of the present invention, a performance test is triggered based on a test plan, and an implementation method for collecting system resource usage data in real time during the test process is: according to the configuration parameters in the test plan, the number of concurrent users and the user startup rate are automatically set, and the performance test is triggered; during the execution of the performance test, the system resource usage data of the host is collected in real time through resource monitoring technology; wherein, the system resource usage data includes at least one of the following: processor utilization, memory utilization and storage resource utilization.

[0084] When triggering a performance test, the number of concurrent users and the user startup rate are automatically set based on the configuration parameters in the test plan. This step involves accessing the performance test platform through automated operations based on the predefined test strategy in the test plan, directly applying the parameter values ​​in the test plan to the test environment without manual input, thus ensuring the accuracy and consistency of parameter settings. The number of concurrent users refers to the total number of users that simulated simultaneously access the interface under test, while the user startup rate refers to the number of users that start per unit time. For example, the number of concurrent users can be set to 1000 and the user startup rate to 100 users per second to simulate the system load pressure in a high-concurrency scenario. Once the settings are completed, the performance test is triggered. The system will simulate a large number of users accessing the interface under test according to the configured parameters, record key metrics such as interface response time and request success rate, and generate corresponding test results. This automated setup and triggering process significantly reduces human intervention, ensuring efficient and repeatable test execution. Secondly, during the performance test execution, resource monitoring technology collects real-time system resource usage data from the host. This process involves using specialized monitoring methods to continuously record the host hardware resource usage at fixed intervals to capture dynamic changes in resource load during the test. Among them, processor utilization reflects the proportion of host computing resources occupied, memory utilization reflects the allocation and use of memory resources, and storage resource utilization reflects the read and write load of the disk or storage device. For example, in a high-concurrency test scenario, you may observe that processor utilization rises rapidly to 80% at the beginning of the test, memory utilization peaks at 90% in the middle of the test, and storage resource utilization remains at a low level. This data can help analyze whether performance issues are related to specific resource bottlenecks. Real-time collection of system resource usage data is usually recorded in time series form, for example, every 10 seconds, to ensure that the data granularity is fine enough to reflect the instantaneous changes and trends in resource usage, providing a reliable basis for subsequent correlation analysis with test result data.

[0085] To illustrate this implementation method more intuitively, assume that the test plan designs a high-load test scenario for a data query interface. The tester automatically sets the number of concurrent users to 2,000 and the user startup rate to 200 users per second according to the test plan, and then triggers the performance test. During the test execution, the host's processor utilization, memory utilization, and storage resource utilization are collected every 5 seconds through resource monitoring technology. It is found that the average processor utilization is 85%, the peak memory utilization is 93%, and the average storage resource utilization is 30%. These data will be fully recorded for subsequent analysis.

[0086] This implementation method automatically sets the number of concurrent users and the user startup rate, avoiding the tediousness and errors of manual configuration, ensuring that the test conditions are consistent with the test plan, and improving test efficiency. Secondly, it collects multi-dimensional system resource usage data such as processor utilization, memory utilization, and storage resource utilization in real time, providing hardware-level support for the comprehensive analysis of performance issues, making up for the shortcomings of traditional testing that only focuses on software indicators, thereby improving the accuracy of problem location and laying a solid data foundation for system optimization.

[0087] In an exemplary embodiment of the present invention, an implementation method of correlating and analyzing system resource usage data with test result data of a performance test to obtain a comprehensive evaluation result is as follows: extracting response time data from the test result data and calculating the statistical value of the response time data; correlating and comparing the statistical value of the system resource usage data with the statistical value of the response time data to obtain a correlation comparison result; judging whether the software performance indicators and / or host resource status meet the preset standards based on the correlation comparison result, and generating a comprehensive evaluation result.

[0088] Extracting response time data from the test result data and calculating response time statistics. This step involves filtering out key metrics related to interface response speed from the test result data generated by the performance test, namely response time data. This typically includes the time interval between each request initiation and the receipt of the response. After extraction, the response time data needs to be statistically analyzed and its statistical values ​​calculated, including but not limited to average, maximum, minimum, and extreme value differences, to comprehensively reflect the performance of the tested interface under different load conditions. For example, in a high-concurrency test scenario, the test result data may record the response time when 1,000 virtual users access the tested interface. Calculations show an average response time of 200 milliseconds, a maximum response time of 500 milliseconds, a minimum response time of 100 milliseconds, and an extreme value difference of 400 milliseconds. These statistical values ​​will serve as an important basis for software performance evaluation. Secondly, the statistical values ​​of the system resource usage data are correlated and compared with the statistical values ​​of the response time data to obtain a correlation comparison result. System resource usage data refers to the host resource utilization collected in real time during the performance test, and its statistical values, such as average and peak values, also need to be calculated. For example, during testing, processor utilization might average 85% and peak at 92%, while memory utilization might average 80% and peak at 96%. Correlation analysis involves analyzing the temporal relationship and changing trends between these two statistical values ​​to identify potential correlations between performance issues and resource load. For example, if a peak response time of 500 milliseconds occurs near a peak memory utilization of 96%, it can be inferred that high response times may be related to memory resource constraints. If the extreme response time differences are large and processor utilization remains consistently high, insufficient computing resources may be the cause. The correlation analysis results will detail these correlations and inferred conclusions, providing data support for subsequent assessments. Finally, based on the correlation analysis results, the software performance indicators and / or host resource status are determined to meet pre-set standards, generating a comprehensive evaluation result. These pre-set standards typically include response time thresholds and resource utilization thresholds. For example, the average response time must be less than 250 milliseconds, the extreme difference must be less than 300 milliseconds, the average processor utilization must be less than 90%, and the peak memory utilization must be less than 95%. If all indicators meet the preset standards, the test is considered passed; otherwise, it is considered a failure, and the specific abnormal indicators and possible causes are recorded. For example, if the response time extreme value difference of 400 milliseconds exceeds the threshold of 300 milliseconds, it is considered a failure, and the correlation comparison results show a correlation with the peak memory utilization exceeding the standard. The final comprehensive evaluation result not only includes the overall pass or fail judgment, but also includes detailed indicator data and correlation analysis conclusions, providing a comprehensive basis for subsequent test report generation and system optimization.

[0089] This implementation calculates response time statistics and system resource usage statistics, providing a quantitative basis for performance evaluation. Second, correlation comparison reveals the causal relationship between software performance and hardware resources, making up for the shortcomings of single-dimensional analysis. Finally, the comprehensive evaluation results based on preset standards provide precise guidance for locating performance issues, thereby greatly improving the comprehensiveness and accuracy of the analysis and laying a solid foundation for system optimization.

[0090] In an exemplary embodiment of the present invention, after correlating and analyzing the system resource occupancy data with the test result data of the performance test to obtain a comprehensive evaluation result, an implementation method is: if the statistical value of the response time data falls within a preset response time threshold range, the software performance test is determined to be successful; if the statistical value of the response time data does not fall within the response time threshold range, the software performance test is determined to have failed; and / or, if the statistical value of the system resource occupancy data falls within a preset resource occupancy threshold range, the host resource test is determined to be successful; if the statistical value of the system resource occupancy data does not fall within the resource occupancy threshold range, the host resource test is determined to have failed.

[0091] A judgment is made based on the statistical values ​​of response time data. Response time data is a key performance indicator extracted from the test result data. Its statistical values ​​include average, maximum, minimum, and extreme value difference, reflecting the response speed of the tested interface under different load conditions. If the statistical values ​​of the response time data fall within the preset response time threshold range, the software performance test is considered successful; conversely, if the statistical values ​​do not fall within this range, the software performance test is considered failed. For example, if the preset response time threshold range is an average value less than 250 milliseconds and an extreme value difference less than 300 milliseconds, and the calculated average response time in the actual test is 200 milliseconds (meets the threshold) but the extreme value difference is 400 milliseconds (exceeds the threshold), the software performance test is considered to have failed because not all statistical values ​​fall within the threshold range. This judgment logic ensures a rigorous assessment of software performance; any failure in any metric will affect the final result. Secondly, a similar judgment is made based on the statistical values ​​of system resource usage data. These statistical values ​​typically include average and peak values, reflecting the resource load of the host during the test. If the statistical value of the system resource usage data falls within the preset resource usage threshold range, the host resource test is judged to be successful; otherwise, if the statistical value does not fall within the range, the host resource test is judged to have failed. For example, assuming that the preset resource usage threshold range is that the average processor utilization is less than 90% and the peak memory utilization is less than 95%, and the average processor utilization in the actual test is 85% (meets the threshold), but the peak memory utilization is 96% (exceeds the threshold), then the host resource test is comprehensively judged to have failed because not all statistical values ​​meet the threshold requirements. It is worth noting that in this embodiment, the judgment of software performance testing and host resource testing can be carried out independently or in combination. After the judgment is completed, the conclusion of success or failure and the specific abnormal indicators and exceeded values ​​will be recorded as part of the comprehensive evaluation results, providing a detailed basis for the generation of subsequent test reports, ensuring that testers can clearly understand which indicators do not meet the standards and their specific values.

[0092] This implementation method uses preset threshold ranges to provide an objective standard for judgment, ensuring the consistency and credibility of the evaluation results. Secondly, success or failure judgments are made for software performance and host resource status, respectively, which refines the granularity of problem location and helps to quickly identify specific abnormal indicators. Finally, the flexible "and / or" judgment logic adapts to the needs of different testing scenarios, improves the applicability of the evaluation, and thus provides accurate and comprehensive guidance for system optimization.

[0093] In an exemplary embodiment of the present invention, an implementation method for generating a test report based on the comprehensive evaluation results is: extracting the judgment information of the software performance indicators and / or host resource status in the comprehensive evaluation results, and according to the preset classification rules, the judgment information is associated and mapped with the statistical values ​​of the corresponding response time data and / or the statistical values ​​of the system resource occupancy data to obtain an association mapping result; and constructing a structured test report based on the association mapping result.

[0094] During the data integration stage, it is necessary to extract the judgment information of software performance indicators and / or host resource status in the comprehensive evaluation results. The comprehensive evaluation results are complete evaluation conclusions formed after the preliminary correlation analysis and threshold judgment, which include two parts: software performance indicators (such as the statistical values ​​of response time data and its success or failure judgment) and host resource status (such as the statistical values ​​of system resource usage data and its success or failure judgment). During extraction, you can selectively focus on software performance indicators, host resource status, or both according to the test objectives. For example, in a test that focuses on interface response speed, the judgment information of software performance indicators may be extracted first, while in a test that focuses on system stability, both may be extracted at the same time. After extraction, the judgment information is associated and mapped with the corresponding statistical values ​​of response time data and / or statistical values ​​of system resource usage data according to the preset classification rules to obtain an association mapping result. Preset classification rules refer to the logic for grouping and associating data based on indicator type or judgment status. For example, all indicators judged as "failed" are classified into one category and correspond one-to-one with specific statistical values. Association mapping refers to establishing a logical connection between judgment information and statistical values. For example, the average response time of 200 milliseconds is associated with the judgment of "pass" (because it is less than the preset threshold of 250 milliseconds), and the peak memory utilization of 96% is associated with the judgment of "failure" (because it is greater than the preset threshold of 95%), ensuring that the relationship between data is clear and traceable. The association mapping results will include the judgment status of all key indicators and their corresponding numerical details, providing a structured data foundation for the construction of test reports. Next, a structured test report is constructed based on the association mapping results. A structured test report refers to organizing test data in a hierarchical and modular format, usually including the overall test result judgment (such as pass or fail) and a detailed indicator classification display. For example, a test report might first present an overall judgment of "failure," then list specific indicators and judgment statuses in sub-items in the form of tables or charts, such as an average response time of 200 milliseconds (pass), an extreme response time difference of 400 milliseconds (failure), an average processor utilization of 85% (pass), and a peak memory utilization of 96% (failure), ensuring that testers can quickly understand the overall situation and specific problem points of the test. In addition, the test report can also include a brief conclusion text of the correlation analysis, such as "A large extreme response time difference may be related to excessive peak memory utilization," providing a reference direction for locating performance bottlenecks.

[0095] The extraction and association mapping process of this implementation ensures that key information in the comprehensive evaluation results is not missed, forming a logically clear data relationship; secondly, the structured test report improves the readability and intuitiveness of the data through hierarchical display, enabling testers to quickly identify abnormal indicators; finally, the analysis conclusions contained in the report provide direct guidance for system optimization, thereby effectively supporting the decision-making process for performance improvement and demonstrating the comprehensive value from data integration to report presentation.

[0096] In an exemplary embodiment of the present invention, after generating a test report based on the comprehensive evaluation results, an implementation method is: adjusting the configuration parameters in the test plan according to the test report; re-triggering the performance test based on the adjusted test plan, obtaining new test result data and comprehensive evaluation results, and updating the test report.

[0097] Adjusting the configuration parameters in the test plan based on the test report involves modifying key parameters in the test plan based on performance issues identified in the test report to improve test conditions or simulate scenarios that better reflect real-world requirements. Test reports typically contain information on software performance metrics and host resource status. For example, if response time statistics (such as average and extreme value differences) indicate that certain metrics fall below target, or if system resource usage statistics (such as processor utilization and memory utilization) exceed preset thresholds, this information can guide configuration parameter adjustments. For example, if a test report shows that extreme value differences in response time are excessive and correlated with a high number of concurrent users, the tester might reduce the number of concurrent users in the test plan from 1,000 to 800, or adjust the user startup rate from 100 users per second to 50 users per second to reduce system load. If the report indicates that peak memory utilization exceeds the target, the test case execution ratio in the test plan might be adjusted to reduce the weight of high-memory-consuming cases. By adjusting configuration parameters, the test plan can better adapt to the current system performance or test objectives, ensuring that subsequent testing is more targeted. Next, the performance test is re-triggered based on the adjusted test plan to obtain new test results and comprehensive evaluation results. This step involves re-accessing the performance test platform through automated operations according to the revised test plan, setting new parameters such as the number of concurrent users and user startup rate, executing the performance test, and recording the new test results, including metrics such as response time and request success rate. Simultaneously, system resource usage data is collected in real time during the test. Correlation analysis and threshold determination are then performed on the new data to generate new comprehensive evaluation results to assess whether the adjusted test plan has effectively improved performance issues. Finally, the test report is updated to incorporate the new test results and comprehensive evaluation results, overwriting or supplementing the original content to form the latest performance analysis conclusions. For example, the updated test report may show that the extreme difference in response time has decreased from 400 milliseconds to 280 milliseconds, meeting the preset threshold range and being judged as passing. At the same time, the peak memory utilization has decreased from 96% to 93%, also meeting the threshold requirement, thus verifying the effectiveness of the configuration parameter adjustments.

[0098] This implementation method specifically addresses performance bottlenecks by adjusting configuration parameters based on the test report, improving the adaptability of the test solution. Secondly, re-triggering the performance test and obtaining new data verifies the adjustment effect, ensuring the scientific nature of the improvement measures. Finally, updating the test report provides the latest performance analysis conclusions, providing a dynamic basis for continuous optimization, thereby effectively supporting the iterative improvement of system performance and the improvement of the test process.

[0099] Based on the above description of an embodiment of a performance testing method, the following describes an automated performance testing method. This method, through fully automated testing, improves performance testing efficiency and enables multi-dimensional performance analysis. It is suitable for comprehensively evaluating the interface performance and resource usage of complex systems. The following describes the implementation of this method in detail, combining specific steps.

[0100] Before introducing the specific steps, let's first explain some of the technical terms involved:

[0101] Locust: A Python-based open source performance testing tool that stress tests application programming interfaces by simulating high-concurrent user behavior.

[0102] Selenium: An open source web automation testing framework used to control browsers to automatically complete user interface interactions (such as parameter configuration and test triggering).

[0103] Python: The core development language, used to integrate Locust, Selenium, and data analysis modules.

[0104] Unittest: Python standard unit testing framework that supports assertion mechanism to verify performance thresholds.

[0105] HTMLTestReport: A Python third-party library that generates a structured Hypertext Markup Language (HTML) report of test results, including pass / fail status and detailed metrics.

[0106] Psutil: A Python library that collects real-time utilization data of server CPU, memory, disk, and other resources to correlate software performance with hardware metrics.

[0107] Reference Figure 2 , shows a step flow chart of an automated performance testing method according to an embodiment of the present invention.

[0108] Step 201: Environment deployment and preparation.

[0109] First, we deployed the environment to ensure the test system had the necessary operating conditions. We installed Python 3.8, the core development language, as the foundation for the test tools and scripts. Next, we deployed Locust 2.15, a performance testing tool, to simulate high-concurrency user behavior and stress-test the interface. We also installed Selenium 4.0, a web automation testing framework, to automate the test tool's interface. We also installed Psutil 5.9, a resource monitoring tool, to collect real-time host resource data. Finally, we installed HTMLTestReport, a test report generation library, to consolidate test results and generate structured reports. After the environment was deployed, we prepared the interfaces to be tested, such as a user data query interface, ensuring that it was connected to the test system and could be called normally. This laid the foundation for subsequent test case development.

[0110] Step 202: Test case construction.

[0111] First, encapsulate the interface to be tested into an executable test function. For example, encapsulate the user data query interface into a function that can simulate query requests and receive return results as a test unit (corresponding to a single interface test case). Based on this test function, combine multiple performance test cases through the TaskSet class, including single-user query cases and multi-user concurrent query cases, and define the execution order and execution ratio of the concurrent task set. For example, execute the single-user query case first, and then execute the multi-user concurrent query case, with the ratios of 70% and 30% respectively. Subsequently, inherit the HttpLocust class to configure the concurrency parameters, set the number of virtual users to 1000, and the user startup rate to 100 users per second, and finally form a test plan to provide guidance for subsequent test execution.

[0112] Step 203: Automatically perform performance testing.

[0113] Using Selenium, I launched a browser and accessed the Locust interface. Based on the test plan's configuration parameters, the system automatically set the number of virtual users to 1,000 and the user launch rate to 100 users per second. Clicking "Start" triggered the test. During the test, the system simulated a large number of users accessing the interface under test, recording metrics such as response time and request success rate, and generating a test result data file. Upon completion, the test result file was automatically downloaded to a designated directory, ensuring data availability for subsequent analysis.

[0114] Step 204: collect system resource usage data in real time.

[0115] During performance testing, Psutil collects real-time data on the host's system resource usage. Resource data is recorded every 10 seconds. For example, processor utilization rises to 85% at the beginning of the test, memory utilization peaks at 96% in the middle, and storage resource utilization remains around 30%. This ensures that dynamic changes in resource load are captured, providing hardware-level support for subsequent correlation analysis.

[0116] Step 205: Data analysis and comprehensive evaluation.

[0117] Correlation analysis is performed between test result data and system resource utilization data to generate a comprehensive evaluation result. Response time data is extracted from the test result data and statistical values ​​are calculated, such as an average response time of 200 milliseconds, a maximum response time of 500 milliseconds, a minimum response time of 100 milliseconds, and an extreme difference of 400 milliseconds. System resource utilization statistics are also calculated, such as an average processor utilization of 85% and a peak memory utilization of 96%. Correlating these two statistical values, it is found that the large extreme difference in response time coincides with the peak memory utilization exceeding the standard, suggesting that the performance issue may be related to insufficient memory resources. Based on pre-set criteria, the test is judged as failed if the average response time is less than 250 milliseconds (pass), the extreme difference is less than 300 milliseconds (fail), the average processor utilization is less than 90% (pass), and the peak memory utilization is less than 95% (fail). A comprehensive evaluation result is generated, including detailed indicators and judgment conclusions.

[0118] Step 206: Generate a structured test report.

[0119] Based on the comprehensive evaluation results, a structured test report is generated using HTMLTestReport. This report integrates information on software performance indicators and host resource status, and maps these information to statistical values ​​according to pre-set classification rules. For example, an average response time of 200 milliseconds (pass), an extreme response time difference of 400 milliseconds (failure), an average processor utilization of 85% (pass), and a peak memory utilization of 96% (failure). The test report, constructed in HTML format, clearly displays the overall test result as "failure" and lists detailed data and analytical conclusions for each indicator, such as "A large extreme response time difference may be related to excessive memory utilization."

[0120] Step 207: Software architecture and modular function implementation.

[0121] The implementation of the embodiment of the present invention relies on a specific software architecture and modular functional design to ensure efficient coordination of each step. The architecture includes the following modules: Locust service module, which carries the interface test task set and provides a web configuration interface; Selenium driver module, which is responsible for automating the operation of the web interface, triggering tests and downloading results; data analysis engine, which integrates test result data parsing, Psutil resource collection and assertion logic, and performs correlation analysis and comprehensive evaluation; report generator, which calls HTMLTestReport to generate the final test report. Each module works closely together according to the data flow. For example, after the Locust service module runs the test script, the Selenium driver module automatically triggers the test, the data analysis engine processes the collected data and generates a comprehensive evaluation result, and finally the report generator outputs the test report to ensure seamless connection of the entire process.

[0122] Step 208, optimization iteration.

[0123] Based on the test report, we identified performance bottlenecks and adjusted the configuration parameters in the test plan, such as reducing the number of concurrent users from 1,000 to 800 to reduce system load. We then re-triggered the performance test based on the adjusted test plan, obtained new test results, and conducted a comprehensive evaluation. We found that the extreme response time difference had dropped to 280 milliseconds (a pass), and the peak memory utilization had dropped to 93% (a pass), ultimately resulting in a pass. We then updated the test report, documenting the optimized performance data to provide a basis for continuous system improvement.

[0124] Through the above steps, the embodiment of the present invention realizes the full process of automated performance testing from environment deployment, test case construction, automated test execution, resource data collection, correlation analysis, test report generation to optimization iteration, significantly improving test efficiency, reducing manual intervention by 90%, and increasing the problem location accuracy by 80% through multi-dimensional analysis, providing reliable support for complex system performance optimization.

[0125] It should be noted that for the sake of simplicity, the method embodiments are described as a series of actions. However, those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0126] Reference Figure 3 , shows a structural block diagram of a performance testing system according to an embodiment of the present invention. The performance testing system may specifically include the following modules.

[0127] A test plan construction module 31 is used to encapsulate the interface to be tested into an executable test unit and construct a test plan based on the test unit;

[0128] Resource data collection module 32, used to trigger performance testing based on the test plan and collect system resource usage data in real time during the test process;

[0129] A data association analysis module 33 is used to perform association analysis on the system resource usage data and the test result data of the performance test to obtain a comprehensive evaluation result;

[0130] The test report generating module 34 is configured to generate a test report according to the comprehensive evaluation result.

[0131] In an exemplary embodiment of the present invention, the test scheme construction module 31 includes:

[0132] A test unit determination module, configured to encapsulate the interface to be tested into an executable test function and use the test function as the test unit;

[0133] The test scheme determination module is used to construct multiple performance test cases based on the test function, and define the execution order and execution ratio of the concurrent task sets of the performance test cases to form the test scheme.

[0134] In an exemplary embodiment of the present invention, the resource data collection module 32 includes:

[0135] A performance test trigger module, configured to automatically set the number of concurrent users and the user startup rate according to the configuration parameters in the test plan, and trigger the performance test;

[0136] An occupancy data collection module is used to collect the system resource occupancy data of the host in real time through resource monitoring technology during the execution of the performance test;

[0137] The system resource occupancy data includes at least one of the following: processor utilization, memory utilization, and storage resource utilization.

[0138] In an exemplary embodiment of the present invention, the data association analysis module 33 includes:

[0139] A time statistics calculation module, configured to extract response time data from the test result data and calculate a statistical value of the response time data;

[0140] A correlation result comparison module, configured to compare the statistical value of the system resource usage data with the statistical value of the response time data to obtain a correlation comparison result;

[0141] The evaluation result generating module is used to judge whether the software performance indicators and / or host resource status meet the preset standards based on the correlation comparison results, and generate the comprehensive evaluation result.

[0142] In an exemplary embodiment of the present invention, the system further comprises:

[0143] a software performance test determination module configured to determine, after the data association analysis module 33 performs association analysis on the system resource usage data and the test result data of the performance test to obtain a comprehensive evaluation result, whether the software performance test is successful if the statistical value of the response time data falls within a preset response time threshold range; and whether the software performance test is failed if the statistical value of the response time data does not fall within the response time threshold range;

[0144] And / or, a host resource test judgment module is used to determine that the host resource test is successful if the statistical value of the system resource occupancy data falls within a preset resource occupancy threshold range; if the statistical value of the system resource occupancy data does not fall within the resource occupancy threshold range, determine that the host resource test has failed.

[0145] In an exemplary embodiment of the present invention, the test report generating module 34 includes:

[0146] an association mapping module, configured to extract the determination information of the software performance indicator and / or the host resource status from the comprehensive evaluation result, and associate and map the determination information with the corresponding statistical values ​​of the response time data and / or the statistical values ​​of the system resource usage data according to a preset classification rule to obtain an association mapping result;

[0147] A report generating module is used to construct the structured test report based on the association mapping result.

[0148] In an exemplary embodiment of the present invention, the system further comprises:

[0149] a configuration parameter adjustment module, configured to adjust configuration parameters in the test plan according to the test report after the test report generation module 34 generates a test report according to the comprehensive evaluation result;

[0150] The test report updating module is used to re-trigger the performance test based on the adjusted test plan, obtain new test result data and comprehensive evaluation results, and update the test report.

[0151] As for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0152] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0153] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, apparatus, or computer program products. Thus, embodiments of the present invention may take the form of a fully hardware embodiment, a fully software embodiment, or an embodiment combining software and hardware. Furthermore, embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0154] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the process in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0155] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0156] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0157] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the basic creative concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0158] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.

[0159] The above is a detailed introduction to a performance testing method and a performance testing system provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A performance testing method, characterized in that: The method comprises: Encapsulate the interface to be tested into an executable test unit, and build a test plan based on the test unit; Triggering performance testing based on the test plan and collecting system resource usage data in real time during the test; Correlation analysis is performed on the system resource usage data and the test result data of the performance test to obtain a comprehensive evaluation result; A test report is generated according to the comprehensive evaluation results.

2. The method according to claim 1, characterized in that The step of encapsulating the interface to be tested into an executable test unit and constructing a test plan based on the test unit includes: Encapsulating the interface to be tested into an executable test function, and using the test function as the test unit; A plurality of performance test cases are constructed based on the test function, and the execution order and execution ratio of the concurrent task sets of the performance test cases are defined to form the test plan.

3. The method according to claim 1 or 2, characterized in that The triggering of the performance test based on the test plan and real-time collection of system resource usage data during the test process include: Automatically set the number of concurrent users and the user startup rate according to the configuration parameters in the test plan, and trigger the performance test; During the performance test, the system resource usage data of the host is collected in real time through resource monitoring technology; The system resource occupancy data includes at least one of the following: processor utilization, memory utilization, and storage resource utilization.

4. The method according to claim 1, wherein The correlating analysis of the system resource usage data with the test result data of the performance test to obtain a comprehensive evaluation result includes: Extracting response time data from the test result data, and calculating a statistical value of the response time data; Correlating and comparing the statistical value of the system resource usage data with the statistical value of the response time data to obtain a correlation comparison result; The software performance indicators and / or host resource status are judged according to the correlation comparison results to determine whether they meet the preset standards, and the comprehensive evaluation results are generated.

5. The method according to claim 4, characterized in that After correlating and analyzing the system resource usage data with the test result data of the performance test to obtain a comprehensive evaluation result, the method further includes: If the statistical value of the response time data falls within a preset response time threshold range, the software performance test is determined to be successful; if the statistical value of the response time data does not fall within the response time threshold range, the software performance test is determined to have failed; And / or, if the statistical value of the system resource occupancy data falls within a preset resource occupancy threshold range, the host resource test is determined to be successful; if the statistical value of the system resource occupancy data does not fall within the resource occupancy threshold range, the host resource test is determined to have failed.

6. The method according to claim 4, characterized in that Generating a test report according to the comprehensive evaluation results includes: Extracting the determination information of the software performance indicator and / or the host resource status from the comprehensive evaluation result, and correlating and mapping the determination information with the corresponding statistical values ​​of the response time data and / or the statistical values ​​of the system resource usage data according to a preset classification rule to obtain a correlation mapping result; A structured test report is constructed based on the association mapping result.

7. The method according to any one of claims 1 to 6, characterized in that After generating a test report according to the comprehensive evaluation result, the method further includes: Adjust the configuration parameters in the test plan according to the test report; The performance test is re-triggered based on the adjusted test plan, new test result data and comprehensive evaluation results are obtained, and the test report is updated.

8. A performance testing system, characterized in that: The system comprises: A test plan building module, used to encapsulate the interface to be tested into executable test units and build a test plan based on the test units; A resource data collection module is used to trigger the performance test based on the test plan and collect system resource usage data in real time during the test; A data correlation analysis module is used to perform correlation analysis on the system resource usage data and the test result data of the performance test to obtain a comprehensive evaluation result; A test report generating module is used to generate a test report according to the comprehensive evaluation result.

9. An electronic device, characterized in that: include: one or more processors; and One or more machine-readable media having instructions stored thereon, when executed by the one or more processors, cause the electronic device to perform the performance testing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer program stored therein enables the processor to execute the performance testing method according to any one of claims 1 to 7.

Images