System performance test method and device, computer equipment and readable storage medium

Abstract

The invention relates to a system performance testing method and device, computer equipment and a readable storage medium, and relates to the technical field of computer performance testing. The method comprises the following steps: determining an initial pressure value and a test script of a to-be-tested system; testing the performance of the to-be-tested system under the initial pressure value through the test script to obtain performance index data of the to-be-tested system under the initial pressure value; increasing the initial pressure value to obtain a new pressure value, and returning to the step of testing the performance of the to-be-tested system under the initial pressure value through the test script until an iteration ending condition is met, so as to obtain performance index data of the to-be-tested system under the plurality of pressure values; and according to the performance index data under each pressure value, determining the critical pressure of the to-be-tested system entering the performance bottleneck period. By adopting the method, the test efficiency can be improved.

Description

Technical Field

[0001] This application relates to the technical field of computer performance testing, and particularly to a system performance testing method, device, computer device, and readable storage medium. Background Art

[0002] Computer systems usually need to perform performance testing. Performance testing can help identify stability issues of the system under high load or long-term operation, so as to ensure that the system can operate normally under various conditions. Traditional stress testing methods often rely on the experience of test technicians, and it is impossible to effectively estimate and select the stress value for system performance iterative testing relying on the experience of technicians, resulting in excessive time consumption due to human intervention during the testing process. Therefore, the system performance testing method in the related technology has the problem of low testing efficiency. Summary of the Invention

[0003] Based on this, it is necessary to provide a system performance testing method, device, computer device, computer-readable storage medium, and computer program product that can improve testing efficiency for the above technical problems.

[0004] In a first aspect, this application provides a system performance testing method, including:

[0005] Determine the initial pressure value and test script for the system under test;

[0006] Test the performance of the system under test at the initial pressure value through the test script to obtain performance index data of the system under test at the initial pressure value;

[0007] Increase the initial pressure value to obtain a new pressure value, and return to the step of testing the performance of the system under test at the initial pressure value through the test script until the iteration end condition is reached, to obtain performance index data of the system under test at multiple pressure values;

[0008] Determine the critical pressure at which the system under test enters the performance bottleneck period according to the performance index data at each pressure value.

[0009] In one embodiment, the increasing the initial pressure value to obtain a new pressure value includes:

[0010] Obtain a preset pressure increment and the current number of tests;

[0011] Increase the initial pressure value according to the pressure increment and the number of tests to obtain a new pressure value.

[0012] In one embodiment, increasing the initial pressure value to obtain a new pressure value includes:

[0013] Determining the cost value of the system under test at the initial pressure value according to the performance index data of the system under test at the initial pressure value and a preset cost model;

[0014] Increasing the cost value of the system under test at the initial pressure value according to a preset cost increment to obtain a new cost value;

[0015] Determining the new pressure value after increasing the initial pressure value according to the new cost value.

[0016] In one embodiment, determining the critical pressure at which the system under test enters the performance bottleneck period according to the performance index data at each pressure value includes:

[0017] Respectively determining the total expected cost value of the system under test at each pressure value according to the performance index data at each pressure value;

[0018] Determining the critical pressure at which the system under test enters the performance bottleneck period according to the total expected cost value at each pressure value.

[0019] In one embodiment, the performance index data includes index values under multiple performance indexes. The step of respectively determining the total expected cost value of the system under test at each pressure value according to the performance index data at each pressure value includes:

[0020] For each pressure value, inputting the performance index data at the pressure value into a preset cost model to obtain the cost value of each performance index at the pressure value;

[0021] Performing weighted summation on the cost values of each performance index according to the preset weight of each performance index to obtain the total expected cost value at the pressure value.

[0022] In one embodiment, the step of determining the critical pressure at which the system under test enters the performance bottleneck period according to the total expected cost value at each pressure value includes:

[0023] Determining the change trend of the total expected cost value as the pressure value increases according to the total expected cost value at each pressure value;

[0024] Determining the inflection point of the total expected cost value according to the change trend of the total expected cost value as the pressure value increases, and using the pressure value corresponding to the inflection point as the critical pressure at which the system under test enters the performance bottleneck period.

[0025] Second aspect, the present application also provides a system performance testing device, including:

[0026] A data determination module, configured to determine an initial pressure value and a test script for the system to be tested;

[0027] An initial test module, configured to test the performance of the system to be tested at the initial pressure value through the test script, and obtain performance index data of the system to be tested at the initial pressure value;

[0028] An iterative test module, configured to increase the initial pressure value to obtain a new pressure value, and return to the step of testing the performance of the system to be tested at the initial pressure value through the test script until an iteration end condition is reached, so as to obtain performance index data of the system to be tested at multiple pressure values;

[0029] A bottleneck determination module, configured to determine a critical pressure at which the system to be tested enters a performance bottleneck period according to the performance index data at each of the pressure values.

[0030] Third aspect, the present application also provides a computer device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0031] Determine an initial pressure value and a test script for the system to be tested;

[0032] Test the performance of the system to be tested at the initial pressure value through the test script, and obtain performance index data of the system to be tested at the initial pressure value;

[0033] Increase the initial pressure value to obtain a new pressure value, and return to the step of testing the performance of the system to be tested at the initial pressure value through the test script until an iteration end condition is reached, so as to obtain performance index data of the system to be tested at multiple pressure values;

[0034] Determine a critical pressure at which the system to be tested enters a performance bottleneck period according to the performance index data at each of the pressure values.

[0035] Fourth aspect, the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0036] Determine an initial pressure value and a test script for the system to be tested;

[0037] Test the performance of the system to be tested at the initial pressure value through the test script, and obtain performance index data of the system to be tested at the initial pressure value;

[0038] Increase the initial pressure value to obtain a new pressure value, and return to the step of testing the performance of the system under test at the initial pressure value through the test script until the iteration end condition is reached, and obtain the performance index data of the system under test at multiple pressure values;

[0039] Determine the critical pressure at which the system under test enters the performance bottleneck period according to the performance index data at each of the pressure values.

[0040] In a fifth aspect, the present application also provides a computer program product, including a computer program, which when executed by a processor implements the following steps:

[0041] Determine the initial pressure value and test script for the system under test;

[0042] Test the performance of the system under test at the initial pressure value through the test script to obtain the performance index data of the system under test at the initial pressure value;

[0043] Increase the initial pressure value to obtain a new pressure value, and return to the step of testing the performance of the system under test at the initial pressure value through the test script until the iteration end condition is reached, and obtain the performance index data of the system under test at multiple pressure values;

[0044] Determine the critical pressure at which the system under test enters the performance bottleneck period according to the performance index data at each of the pressure values.

[0045] For the above system performance testing method, device, computer device, computer-readable storage medium, and computer program product, the method determines the initial pressure value and test script for the system under test; tests the performance of the system under test at the initial pressure value through the test script to obtain the performance index data of the system under test at the initial pressure value; increases the initial pressure value to obtain a new pressure value, and returns to the step of testing the performance of the system under test at the initial pressure value through the test script until the iteration end condition is reached, and obtains the performance index data of the system under test at multiple pressure values. It does not rely on the experience of technicians to select the pressure values for testing, but instead performs iterative testing by gradually increasing the pressure values in a preset manner by the server. According to the performance index data at each pressure value, it determines the critical pressure at which the system under test enters the performance bottleneck period, realizing the automated processing of the iterative testing of the system under test, thereby improving the testing efficiency. Description of the Drawings

[0046] To more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments of the present application or related technologies. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0047] Figure 1 It is a schematic flowchart of a system performance testing method in an embodiment;

[0048] Figure 2 It is a schematic flowchart of a critical pressure determination step in an embodiment;

[0049] Figure 3 It is a schematic flowchart of a system performance testing method in another embodiment;

[0050] Figure 4 It is a structural block diagram of a system performance testing device in an embodiment;

[0051] Figure 5 It is an internal structure diagram of a computer device in an embodiment. Specific embodiments

[0052] In order to make the purpose, technical solutions and advantages of the present application more clear, the following further details the present application in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0053] In one embodiment, as Figure 1 shown, a system performance testing method is provided. In this embodiment, it is exemplified that the method is applied to a server. It can be understood that the method can also be applied to a terminal, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0054] Step S102, determine the initial pressure value and test script for the system to be tested.

[0055] Among them, the system to be tested can be a computer system that needs to perform performance testing. The computer system is a collection composed of hardware and software, and is used to receive, process and store information.

[0056] Among them, the initial pressure value can be the load applied to the system during testing, such as the number of users operating simultaneously, or the number of transactions or requests processed within a specific time, etc.

[0057] Among them, the test script can be an automated test tool, which is used to simulate multi-user concurrent operations for the system under test, and record the operation behaviors of users in the system by using the automated test tool; it includes common operations such as user login, page browsing, form submission, and data query.

[0058] Optionally, the server filters out the load when the system under test is operating normally from the historical performance data of the system under test as the initial pressure value, obtains the preset test script and the data required by the test script. The required data includes virtual user data and test case data, and associates the test script with the above data. It can be understood that before the formal test, it is also necessary to verify the test script and conduct a trial run of the test script to verify its correctness and stability. Among them, the virtual user data needs to cover different types of users and operation scenarios, including user names, passwords, and user IDs (Identification). Among them, the test case data needs to cover the main functions and boundary conditions of the system under test, including input data, expected output, and test conditions.

[0059] Step S104, test the performance of the system under test at the initial pressure value through the test script, and obtain the performance index data of the system under test at the initial pressure value.

[0060] Among them, the performance index data can be index values that reflect the performance of the system such as response speed, processing capacity, and stability.

[0061] Optionally, the server calls the test script to test the performance of the system under test at the initial pressure value, and obtains the performance index data of the system under test at the initial pressure value. For example, set the pressure value of the test system at the initial pressure value, run the test script, simulate multi-user concurrent operations on the system under test, and record the performance index data of the system under test.

[0062] Step S106, perform an increasing process on the initial pressure value to obtain a new pressure value, and return to the step of testing the performance of the system under test at the initial pressure value through the test script until the iteration end condition is reached, and obtain the performance index data of the system under test at multiple pressure values.

[0063] Among them, the increasing process can be a processing method of increasing the value, that is, increasing the value of the initial pressure value. The increasing method can be linear or non-linear.

[0064] Among them, the iteration end condition can be that the number of iteration tests is greater than the preset iteration number threshold.

[0065] Optionally, the server processes the initial pressure value to increase it, obtains a new pressure value, returns the steps of testing the performance of the system under test at the initial pressure value through the test script, and uses the new pressure value to test the system under test until the iteration end condition is reached, obtaining the performance index data of the system under test at multiple pressure values.

[0066] Step S108: Determine the critical pressure at which the system under test enters the performance bottleneck period according to the performance index data at each pressure value.

[0067] Among them, the performance bottleneck period may refer to that during the operation of a computer system, the performance limitation of a certain component or resource causes the entire system to be unable to fully utilize its potential processing power, usually affecting the system's response time, processing speed, and overall performance.

[0068] Among them, the critical pressure may be the pressure value corresponding to when the system under test enters the performance bottleneck period.

[0069] Optionally, the server determines the test at which the system under test enters the performance bottleneck period according to the performance index data measured by the system under test at each pressure value, and uses the pressure value corresponding to this test as the critical pressure.

[0070] In the above system performance testing method, the method determines the initial pressure value and test script for the system under test; tests the performance of the system under test at the initial pressure value through the test script to obtain the performance index data of the system under test at the initial pressure value; processes the initial pressure value to increase it to obtain a new pressure value, and returns the step of testing the performance of the system under test at the initial pressure value through the test script until the iteration end condition is reached, obtaining the performance index data of the system under test at multiple pressure values. It does not rely on the experience of technicians to select the pressure value for testing, but iteratively tests by the server gradually increasing the pressure value in a preset manner. According to the performance index data at each pressure value, it determines the critical pressure at which the system under test enters the performance bottleneck period, realizing the automated processing of the iterative testing of the system under test, thereby improving the testing efficiency.

[0071] In an exemplary embodiment, the content of processing the initial pressure value to increase it to obtain a new pressure value in step S106 includes:

[0072] Obtain the preset pressure increment and the current test times; process the initial pressure value to increase it according to the pressure increment and the test times to obtain a new pressure value.

[0073] Among them, the pressure increment may be the part increased based on the initial pressure value as the reference value or the original value.

[0074] Optionally, the server obtains a preset pressure increment and the current number of tests, and increases the initial pressure value according to the pressure increment and the number of tests to obtain a new pressure value. The corresponding expression includes:

[0075]

[0076] Among them, is the new pressure value corresponding to the i-th test, is the initial pressure value, is the pressure increment, is the current number of tests.

[0077] In this embodiment, by means of linear increment, the initial pressure value is increased during each test to obtain a new pressure value for iterative testing, realizing that the server automatically selects the pressure value for each iterative test, thereby further improving the test efficiency, and linearly increasing the pressure value of each test can make the test results more coherent.

[0078] In an exemplary embodiment, the content of increasing the initial pressure value in step S106 to obtain a new pressure value includes:

[0079] Determine the cost value of the system under test at the initial pressure value according to the performance index data of the system under test at the initial pressure value and a preset cost model; increase the cost value of the system under test at the initial pressure value according to a preset cost increment to obtain a new cost value; determine the new pressure value after increasing the initial pressure value according to the new cost value.

[0080] Among them, the preset cost model can be a preset mathematical function model for calculating the cost value, and the cost model is used to quantify the resource consumption or business loss of the system under test under different pressure values.

[0081] Optionally, the server inputs the performance index data of the system under test at the initial pressure value into a preset cost model to obtain the cost values of multiple performance indicators corresponding to the performance index data of the system under test at the initial pressure value, add the preset cost increment and the cost value at the initial pressure value, and increase each cost value of the system under test at the initial pressure value in this way to obtain a new cost value. According to each new cost value and the preset cost model, inversely calculate the sub-pressure value corresponding to each performance indicator, and obtain the new pressure value according to the sum value of each sub-pressure value. The corresponding expression is:

[0082]

[0083] Among them, represents the new pressure value corresponding to the (i + 1)-th test, represents at the pressure value The cost value corresponding to the x-th performance metric below indicating the cost increment indicating the fixed cost value indicating the preset cost coefficient

[0084] In this embodiment, by iteratively increasing the cost value to inversely deduce the new pressure value, the coherence and predictability of the test process are ensured, the iteration process of the pressure value is accelerated and the convergence speed is increased, thereby reducing the number of tests and further improving the test efficiency.

[0085] In an exemplary embodiment, as Figure 2 shown, step S108 determines the critical pressure at which the system under test enters the performance bottleneck period according to the performance metric data at each pressure value, including:

[0086] Step S202, respectively determine the total cost expectation value of the system under test at each pressure value according to the performance metric data at each pressure value;

[0087] Among them, the total cost expectation value can be the probability average estimate of the total cost value under uncertain environment considering different test pressures or fluctuations of different performance metric data.

[0088] Optionally, the server respectively determines the cost value of the system under test at each pressure value according to the performance metric data at each pressure value, and further determines the total cost expectation value of the system under test at each pressure value according to the cost value at each pressure value.

[0089] Step S204, determine the critical pressure at which the system under test enters the performance bottleneck period according to the total cost expectation value at each pressure value.

[0090] Optionally, the server uses statistical methods to analyze the change trend of the total cost expectation value at each pressure value, and determines the critical pressure at which the system under test enters the performance bottleneck period according to the change trend.

[0091] In this embodiment, by locating the system performance at each pressure value, identifying the starting point of performance decay, and calculating the total cost expectation value to eliminate the contingency of single test (such as network jitter, instantaneous load imbalance), the robustness of the test results is enhanced. In addition, by using mathematical methods (such as derivative change, curvature analysis) to replace subjective experience judgment, human error is avoided and the test accuracy is improved.

[0092] In an exemplary embodiment, the performance metric data includes the metric values under multiple performance metrics. Step S202 respectively determines the total cost expectation value of the system under test at each pressure value according to the performance metric data at each pressure value, including:

[0093] For each pressure value, input the performance index data corresponding to the pressure value into a preset cost model to obtain the cost value of each performance index at the pressure value; perform weighted summation on the cost values of each performance index according to the preset weight of each performance index to obtain the expected total cost at the pressure value.

[0094] Among them, the performance index data includes index values under multiple performance indexes, such as including hardware performance indexes, server performance indexes, middleware performance indexes, application program performance indexes, and network performance indexes. Among them, the performance indexes of the hardware in the system to be tested include CPU usage rate, memory usage rate, and hard disk read / write speed: the performance indexes of the server include thread pool usage rate, number of connections, and request processing time; the performance indexes of the middleware include message queue length, transaction processing time, and cache hit rate; the performance indexes of the application program include code call time, database query time, and memory leak rate; the performance indexes of the network include network bandwidth usage rate, network latency, and packet loss rate.

[0095] Among them, the preset weight can be set according to the importance of each performance index in the actual test requirements.

[0096] Optionally, for each pressure value, the server inputs the performance index data corresponding to the pressure value into a preset cost model to obtain the cost value of each performance index at the pressure value. The corresponding expression includes:

[0097]

[0098] Among them, represents the cost value corresponding to the x-th performance index at the pressure value L, is a preset cost coefficient, represents the index value of the x-th performance index at the pressure value L, and C is a fixed cost, which is a constant term.

[0099] Furthermore, the server performs weighted summation on the cost values of each performance index according to the preset weight of each performance index to obtain the expected total cost at the pressure value. The corresponding expression is:

[0100]

[0101] Among them, is the expected total cost of the system to be tested at the pressure value L, is the weight of the x-th performance index, is the cost value corresponding to the x-th performance index at the pressure value L.

[0102] In this embodiment, by determining the expected total cost of the system to be tested at each pressure value, it lays a foundation for subsequent determination of the critical pressure of the performance bottleneck period.

[0103] In an exemplary embodiment, step S204 determines the critical pressure at which the system under test enters the performance bottleneck period according to the expected total cost at each pressure value, including:

[0104] Determine the changing trend of the expected total cost as the pressure value increases according to the expected total cost at each pressure value; determine the inflection point of the expected total cost according to the changing trend of the expected total cost as the pressure value increases, and use the pressure value corresponding to the inflection point as the critical pressure at which the system under test enters the performance bottleneck period.

[0105] Among them, the inflection point can be a point where the concavity and convexity of the curve change in calculus. At these points, the second derivative is zero or does not exist, and the shape of the curve changes from concave to convex, or vice versa.

[0106] Optionally, the server arranges the expected total costs at each pressure value in ascending order of the pressure value, and uses statistical methods such as calculating the mean, variance of the expected total cost or forming a trend line for analysis to obtain the changing trend of the expected total cost as the pressure value increases. The server further determines the inflection point of the expected total cost according to the changing trend of the expected total cost as the pressure value increases, and uses the pressure value corresponding to the inflection point as the critical pressure at which the system under test enters the performance bottleneck period.

[0107] In this embodiment, by gradually increasing the test pressure, recording and analyzing the performance of the system under different loads, and comprehensively evaluating the performance of the system, it helps to identify the performance bottleneck of the system under high load. By calculating the cost function and the expected total cost, the performance consumption of the system under different pressures is quantified, providing an intuitive data basis for subsequent statistical methods (such as mean, variance, trend line) to analyze the changing trend, so as to be able to intuitively and accurately identify the inflection point of the expected total cost at which the system under test enters the performance bottleneck period and obtain the corresponding critical pressure.

[0108] In an exemplary embodiment, after step S108 determines the critical pressure at which the system under test enters the performance bottleneck period according to the expected total cost at each pressure value, it further includes:

[0109] Hierarchically divide each component in the system under test to obtain the corresponding level of each component; in the order from bottom to top of the levels, perform performance analysis on the components of each level in turn to determine the components that cause the performance bottleneck in the system under test.

[0110] Among them, the component can refer to various hardware and software components that make up a computer.

[0111] Optionally, the server hierarchically divides each component in the system under test, from bottom to top, including hardware, server, middleware, application, and network in sequence, to obtain the corresponding layer for each component. The server further analyzes the performance of the components at each layer in sequence from bottom to top according to the layer order using a performance testing tool, determines the components that cause performance bottlenecks in the system under test, and formulates corresponding optimization solutions, such as hardware upgrade, server configuration optimization, middleware tuning, code optimization, network optimization, etc. It should be noted that after the server executes the corresponding optimization solution, the performance test of the system under test can be performed again to find the critical pressure at the next performance bottleneck period until the performance bottleneck positioning and optimization of all components in the system under test are completed.

[0112] In this embodiment, the performance testing (load testing, stress testing) method is deduced and designed based on the basic principles of the system architecture. Based on stress testing, since only one performance bottleneck is discovered and solved in each performance test, an iterative testing framework is proposed. Starting from the bottom-level system components, if this system component is tested and found to have no performance bottleneck, then in subsequent tests, if there is a performance bottleneck, it must not be caused by this component, but by the upper-level system components. "From bottom to top" in this embodiment means that the system under test is hierarchically divided from bottom to top into: hardware, server, middleware, application, and network. And it is pointed out that the flow direction of the performance bottleneck is one-way, that is, problems in the lower-level components will affect the performance of the upper-level components, while problems in the upper-level components will not affect the performance of the lower-level components. This idea of hierarchical positioning can help testers avoid the confusion caused by the mutual influence of components at different levels and make the positioning of performance bottlenecks more organized.

[0113] In an exemplary embodiment, as Figure 3 shown, another system performance testing method is provided, including:

[0114] Step 1, test script preparation and data preparation.

[0115] Exemplarily, use an automated testing tool to record the behavior of users browsing the system, edit the recorded script, and develop a test script that meets the requirements of performance testing. Prepare the data required for the test script, including virtual user data and test case data, and associate the data with the test script.

[0116] Specifically, use an automated testing tool to record the operation behaviors of users in the system; including common operations such as user login, page browsing, form submission, and data query. Edit the recorded script to ensure it meets the requirements of performance testing. Add necessary parameter settings, such as the data entered by users and the requested URL (Uniform Resource Locator). Ensure that the script can simulate multi-user concurrent operations, and set appropriate numbers of concurrent users and execution times. Run the edited script to verify its correctness and stability. Ensure that the script can execute normally and accurately simulate user behaviors. Prepare virtual user data: Create virtual user data, including usernames, passwords, and user IDs. Ensure that the virtual user data can cover different types of users and operation scenarios. Prepare test case data, including input data, expected outputs, and test conditions. Ensure that the test case data can cover the main functions and boundary conditions of the system. Associate the virtual user data and test case data with the test script. Ensure that the script can correctly call and use this data during execution.

[0117] Step 2, test scenario design and execution.

[0118] Exemplarily, select an initial pressure value from the historical performance data of the system under test and conduct a preliminary test; specifically, the selection of the initial pressure value should be based on the normal operating load of the system to ensure that the starting point of the test is representative. At each pressure value, record the performance index values of the components in the system under test; these indexes reflect the resource consumption of the system under different loads. For the cost function of each performance index at a specific test pressure (corresponding to the cost model in the above embodiment), in the performance bottleneck location and optimization of the hardware, use the CPU (Central Processing Unit) utilization rate, memory utilization rate, and hard disk read / write rate as performance indexes to calculate the critical point and locate the specific bottleneck point; while in other servers, middleware, applications, and networks, different performance indexes are used. According to the cost function values (cost values) of the performance indexes obtained from the cost function, calculate the total cost expectation value of the system under test under iterative increasing test pressures. The total cost expectation value is used to comprehensively evaluate the overall performance of the system under different pressures. Use statistical methods (such as mean, variance, trend line) to analyze the change trend of the total cost expectation value data, and find the inflection point of the change of the total cost expectation value data. The inflection point corresponds to the critical point when the system under test enters the performance bottleneck period; the test scenario design of this step comprehensively evaluates the performance of the system by gradually increasing the test pressure and recording and analyzing the performance of the system under different loads. This helps to identify the performance bottlenecks of the system under high loads. After that, through comprehensive bottleneck identification and performance evaluation, provide guidance for system optimization. The optimization plan can be adjusted and improved based on the test results for specific performance bottlenecks.

[0119] Among them, by calculating the cost function and the total expected cost, the performance consumption of the quantization system under different pressures is quantified. This provides an intuitive data basis for subsequent statistical methods (such as mean, variance, trend line) to analyze the change trend.

[0120] Among them, for the performance index values of the components in the system to be tested, according to the hierarchical division rules of each system component. Among them, the performance indicators of the hardware in the system to be tested include CPU usage rate, memory usage rate, and hard disk read / write speed; the performance indicators of the server include thread pool usage rate, number of connections, and request processing time; the performance indicators of the middleware include message queue length, transaction processing time, and cache hit rate; the performance indicators of the application program include code call time, database query time, and memory leak rate; the performance indicators of the network include network bandwidth usage rate, network latency, and packet loss rate. Specifically, for the hardware performance indicators: CPU usage rate: A high CPU usage rate may indicate that the system load is too high, which may lead to performance degradation or slow system response. Memory usage rate: A high memory usage rate may lead to system performance degradation or out-of-memory errors. Hard disk read / write speed: A high hard disk read / write speed may lead to I / O bottlenecks and affect system performance. For the server performance indicators: Thread pool usage rate: A high thread pool usage rate may lead to thread contention and performance degradation. Number of connections: A high number of connections may cause server overload and affect the response time. Request processing time: A long request processing time may indicate server performance issues or resource shortages. For the middleware performance indicators: Message queue length: It represents the number of messages waiting to be processed in the message queue. A long message queue may lead to delays and performance degradation. Transaction processing time: It represents the time required for the middleware to process each transaction. A long transaction processing time may indicate middleware performance issues or resource shortages. Cache hit rate: It represents the proportion of cache requests that are hit. A low cache hit rate may lead to frequent database accesses and affect performance. For the application program performance indicators: Code call time: It represents the execution time of each code segment in the application program. A long code call time may indicate low code efficiency or resource contention. Database query time: It represents the time required for the application program to execute database queries. A long database query time may indicate database performance issues or low query efficiency. Memory leak rate: It represents the proportion of memory that is not released in the application program. A high memory leak rate may lead to out-of-memory and performance degradation. For the network performance indicators: Network bandwidth usage rate: It represents the usage of network bandwidth, usually expressed as a percentage. A high network bandwidth usage rate may lead to network congestion and performance degradation. Network latency: It represents the time for a data packet to be transmitted in the network. A high network latency may lead to increased response time and degraded user experience. Packet loss rate: It represents the proportion of data packets lost during network transmission. A high packet loss rate may lead to communication failures and performance degradation.

[0121] By monitoring and analyzing the above performance metrics, the performance of the system under different loads can be comprehensively evaluated, performance bottlenecks can be identified and resolved, and the overall performance and stability of the system can be improved.

[0122] The method of iteratively increasing the test pressure is the linear increment method; the specific steps include: setting a fixed increment ΔL based on the initial pressure value, and increasing the pressure value by ΔL in each iteration; the method of iteratively increasing the test pressure is the cost increment method; the specific steps include: setting a cost increment ΔC, and adjusting the pressure value according to the cost increment in each iteration; according to the cost function under the new pressure value, determining the new pressure value through reverse calculation.

[0123] Specifically, the cost increment method can analyze the change trend of the total cost expectation value data according to statistical methods (such as mean, variance, trend line) by setting the cost increment, and determine a cost increment ΔC according to the curvature of the change trend.

[0124] This method ensures the coherence and predictability of the test process, speeds up the convergence rate of the pressure iteration process, reduces the number of tests, and improves the test efficiency.

[0125] Step 3, perform performance bottleneck location.

[0126] Exemplarily, according to the hierarchical division rules of the components in the system, a bottom-up performance bottleneck location method is adopted to check all system components one by one; the system components are divided from bottom to top, including hardware, server, middleware, application program, and network in sequence; use a performance analysis tool to monitor and hierarchically analyze the system performance; according to the results of the hierarchical analysis, locate the performance bottleneck points of each system component.

[0127] It should be noted that the performance testing (load testing, stress testing) method is deduced and designed based on the basic principles of the system architecture. The present invention is based on stress testing. Since each performance test only discovers and resolves one performance bottleneck of it, an iterative test framework is proposed. Starting from the underlying system components, if this system component is tested and there is no performance bottleneck, then in subsequent tests, if there is a performance bottleneck, it must not be caused by this component, but by the upper-level system components.

[0128] "Bottom-up" in this step means that the system to be tested is hierarchically divided from bottom to top into: hardware, server, middleware, application program, and network. And it is pointed out that the flow direction of the performance bottleneck is one-way, that is, problems in the lower-level components will affect the performance of the upper-level components, while problems in the upper-level components will not affect the performance of the lower-level components. This idea of hierarchical location can help testers avoid the confusion caused by the mutual influence of components at different levels and make the location of performance bottlenecks more organized.

[0129] The specific steps for monitoring and hierarchical analysis of system performance using performance analysis tools and log analysis tools include: installing and configuring performance analysis tools to ensure compatibility with the system under test and correct integration into the system under test; using performance analysis tools to monitor the performance metrics of the system under test in real time; and by hierarchically and individually analyzing the performance metrics of the hardware, servers, middleware, applications, and network in the system under test, thereby locating the performance bottleneck points of each system component.

[0130] Specifically, different performance analysis tools can be used to view the performance of components at different levels (view the performance of hardware, middleware, applications, servers, and networks). Based on the results of hierarchical analysis, locate the specific performance bottleneck points. Determine the root cause of the bottleneck, such as insufficient resources, improper configuration, or low code efficiency. Then proceed with the subsequent steps, and formulate corresponding optimization plans for the identified performance bottlenecks. The optimization plans include hardware upgrades, server configuration optimization, middleware tuning, code optimization, and network optimization. Implement the optimization plans to optimize the system. Then conduct performance testing again to verify whether the optimization effect meets the expectations.

[0131] Step 4: Optimization. Formulate corresponding optimization plans for the located performance bottlenecks; optimize the system according to the optimization plans, and verify whether the optimization effect meets the expectations.

[0132] Step 5: Loop testing: Repeat Steps 2 to 4, conduct performance testing again to find the critical point of the next performance bottleneck period of the system under test; loop in this way until the performance bottleneck location and optimization of all system components are completed. Design a closed-loop plan for testing, locating, optimizing, and retesting the system performance bottleneck based on the hierarchical division rules; gradually optimize the system performance until the performance bottleneck location and optimization of all system components are completed. Ensure a comprehensive evaluation and continuous optimization of the system performance, and improve the overall performance and stability of the system.

[0133] In this embodiment, the test scenario design uses the linear increment method or the cost increment method to iteratively increase the test pressure, calculates the cost function and the total cost expectation value through various performance metrics, uses statistical methods to analyze the change trend of the total cost expectation value data, and judges and finds the performance bottleneck. Ensure the coherence and predictability of the test process, accelerate the convergence speed of the pressure iteration process to reduce the number of tests and improve the test efficiency. According to the hierarchical division rules of system components, use a bottom-up method to check all system components one by one, and use performance analysis tools for monitoring and hierarchical analysis to locate the performance bottleneck points. Ensure the comprehensiveness of the test. The present invention designs a closed-loop plan for testing, locating, optimizing, and retesting the system performance bottleneck based on the hierarchical division rules, ensures a comprehensive evaluation and continuous optimization of the system performance, and improves the overall performance and stability of the system.

[0134] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are sequentially shown according to the indications of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this document, the execution of these steps has no strict order limitation, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0135] Based on the same inventive concept, the embodiments of the present application also provide a system performance testing device for implementing the system performance testing method described above. The solution provided by this device to solve the problem is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the system performance testing device provided below can refer to the limitations on the system performance testing method in the above text, and will not be repeated here.

[0136] In an exemplary embodiment, as Figure 4 shown, a system performance testing device 400 is provided, including: a data determination module 401, an initial testing module 402, an iterative testing module 403, and a bottleneck determination module 404, where:

[0137] The data determination module 401 is configured to determine an initial pressure value and a test script for the system to be tested.

[0138] The initial testing module 402 is configured to test the performance of the system to be tested at the initial pressure value through the test script, and obtain performance index data of the system to be tested at the initial pressure value.

[0139] The iterative testing module 403 is configured to increase the initial pressure value to obtain a new pressure value, and return to the step of testing the performance of the system to be tested at the initial pressure value through the test script until the iteration end condition is reached, and obtain performance index data of the system to be tested at multiple pressure values.

[0140] The bottleneck determination module 404 is configured to determine the critical pressure at which the system to be tested enters the performance bottleneck period according to the performance index data at each pressure value.

[0141] Further, in one embodiment, the iterative testing module 403 is further configured to obtain a preset pressure increment and the current number of test times; and increase the initial pressure value according to the pressure increment and the number of test times to obtain a new pressure value.

[0142] Further, in one embodiment, the iterative testing module 403 is further configured to determine the cost value of the system under test at the initial pressure value according to the performance index data of the system under test at the initial pressure value and a preset cost model; increase the cost value of the system under test at the initial pressure value according to a preset cost increment to obtain a new cost value; and determine a new pressure value after increasing the initial pressure value according to the new cost value.

[0143] Further, in one embodiment, the bottleneck determination module 404 is further configured to determine the total expected cost of the system under test at each pressure value according to the performance index data at each pressure value; and determine the critical pressure at which the system under test enters the performance bottleneck period according to the total expected cost at each pressure value.

[0144] Further, in one embodiment, for each pressure value, the bottleneck determination module 404 is further configured to input the performance index data at the pressure value into a preset cost model to obtain the cost value of each performance index at the pressure value; and perform weighted summation on the cost values of each performance index according to the preset weight of each performance index to obtain the total expected cost at the pressure value.

[0145] Further, in one embodiment, the bottleneck determination module 404 is further configured to determine the change trend of the total expected cost as the pressure value increases according to the total expected cost at each pressure value; determine the inflection point of the total expected cost according to the change trend of the total expected cost as the pressure value increases; and use the pressure value corresponding to the inflection point as the critical pressure at which the system under test enters the performance bottleneck period.

[0146] Further, in one embodiment, the bottleneck determination module 404 is further configured to hierarchically divide each component in the system under test to obtain the corresponding level of each component; and sequentially perform performance analysis on the components at each level in the order from bottom to top to determine the component that causes the performance bottleneck in the system under test.

[0147] Each module in the above system performance testing device 400 can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor of the computer device in the form of hardware or independent of it, or stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0148] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in Figure 5 shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as initial pressure values, test script-related data, performance metric data, and new pressure values. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through a network connection. When the computer program is executed by the processor, it implements a system performance test method.

[0149] Those skilled in the art can understand that Figure 5 the structure shown in is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0150] In an embodiment, a computer device is further provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.

[0151] In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0152] In an embodiment, a computer program product is provided, including a computer program. When the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0153] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0154] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in this application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, artificial intelligence (AI) processors, etc., without limitation.

[0155] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope recorded in this application.

[0156] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several variations and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A system performance testing method, characterized in that, The method includes: Determine an initial pressure value for the system under test and a test script; Test the performance of the system under test at the initial pressure value through the test script to obtain performance index data of the system under test at the initial pressure value; Increase the initial pressure value to obtain a new pressure value, and return to the step of testing the performance of the system under test at the initial pressure value through the test script until the iteration end condition is reached, to obtain performance index data of the system under test at multiple pressure values; Determine the critical pressure at which the system under test enters the performance bottleneck period according to the performance index data at each of the pressure values.

2. The method according to claim 1, characterized in that, The increasing the initial pressure value to obtain a new pressure value includes: Obtain a preset pressure increment and the current number of tests; Increase the initial pressure value according to the pressure increment and the number of tests to obtain a new pressure value.

3. The method according to claim 1, wherein The increasing the initial pressure value to obtain a new pressure value includes: Determine the cost value of the system under test at the initial pressure value according to the performance index data of the system under test at the initial pressure value and a preset cost model; Increase the cost value of the system under test at the initial pressure value according to a preset cost increment to obtain a new cost value; Determine the new pressure value after increasing the initial pressure value according to the new cost value.

4. The method according to claim 1, characterized in that, The determining the critical pressure at which the system under test enters the performance bottleneck period according to the performance index data at each of the pressure values includes: Respectively determine the total cost expectation value of the system under test at each of the pressure values according to the performance index data at each of the pressure values; Determine the critical pressure at which the system under test enters the performance bottleneck period according to the total cost expectation values at each of the pressure values.

5. The method according to claim 4, wherein The performance index data includes index values under multiple performance indexes. The respectively determining the total cost expectation value of the system under test at each of the pressure values according to the performance index data at each of the pressure values includes: For each pressure value, input the performance index data at the pressure value into a preset cost model to obtain the cost value of each performance index at the pressure value; Perform weighted summation on the cost values of each performance index according to the preset weight of each performance index to obtain the total cost expectation value at the pressure value.

6. The method according to claim 4, characterized in that The determining the critical pressure at which the system under test enters the performance bottleneck period according to the total cost expectation values at each of the pressure values includes: Determine the change trend of the total cost expectation value as the pressure value increases according to the total cost expectation values at each of the pressure values; Determine the inflection point of the total cost expectation value according to the change trend of the total cost expectation value as the pressure value increases, and use the pressure value corresponding to the inflection point as the critical pressure at which the system under test enters the performance bottleneck period.

7. The method according to claim 1, characterized in that, After the determining the critical pressure at which the system under test enters the performance bottleneck period according to the total cost expectation values at each of the pressure values, it further includes: Hierarchically divide each component in the system under test to obtain the level corresponding to each component; In the order from bottom to top according to the levels, perform performance analysis on the components of each level in turn to determine the components in the system under test that cause performance bottlenecks.

8. A system performance testing device, characterized in that, The device includes: A data determination module, configured to determine an initial pressure value and a test script for the system under test; An initial test module, configured to test the performance of the system under test at the initial pressure value through the test script to obtain performance index data of the system under test at the initial pressure value; An iterative test module, configured to increase the initial pressure value to obtain a new pressure value, and return to the step of testing the performance of the system under test at the initial pressure value through the test script until an iteration end condition is reached, to obtain performance index data of the system under test at multiple pressure values; A bottleneck determination module, configured to determine a critical pressure at which the system under test enters a performance bottleneck period according to the performance index data at each of the pressure values.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images