Reference performance test method, device, equipment, medium and product

Abstract

The invention discloses a benchmark performance testing method and device, electronic equipment, a storage medium and a computer program product, relates to the technical field of automatic testing, is applied to a distributed cluster, and comprises the steps of generating a test case according to a received case factor, executing a benchmark test in the test case for any test case according to a preset concurrent parameter, and executing the benchmark performance of the test case according to the preset concurrent parameter. Obtaining a test execution record of the test case; performing iterative algorithm analysis on the test execution record, and determining a local optimal weight corresponding to each dimension index in the test execution record; and comparing and analyzing the benchmark performance of the distributed cluster from a plurality of dimension indexes based on the local optimal weight. The technical problem of low test efficiency is solved.

Description

Technical Field

[0001] The present application relates to the field of automated testing technology, and in particular to a benchmark performance testing method, device, electronic device, storage medium and computer program product. Background Art

[0002] With the rapid development of Internet technology, distributed clusters have become key components of many systems, and their performance is directly related to core indicators such as system concurrency, response speed, and traffic. Taking the distributed cluster as an example, with the rise of cloud computing, the deployment methods and underlying resources of Redis services are becoming increasingly diverse, and performance has become difficult to predict. Therefore, it is particularly important to benchmark the performance of Redis services.

[0003] Although the demand for benchmark performance testing of Redis services is growing, the current testing methods have many shortcomings. The testing process mainly relies on manual operations by testers, from designing cases to logging into the server, to executing commands, collecting and analyzing results. The entire process is cumbersome and inefficient. In addition, the existing testing tools have complex parameter commands, high learning costs, and low universality, which further increases the difficulty of testing. Therefore, the current evaluation of the benchmark performance of distributed clusters has the problem of low test efficiency. Summary of the invention

[0004] The main purpose of this application is to provide a benchmark performance testing method, device, electronic device, storage medium and computer program product, aiming to solve the technical problem of low testing efficiency.

[0005] To achieve the above objectives, the present application proposes a benchmark performance testing method, which is applied to a distributed cluster. The benchmark performance testing method includes:

[0006] Generate a test case according to the received case factors, and for any of the test cases, execute a benchmark test in the test case according to preset concurrency parameters to obtain a test execution record of the test case;

[0007] Performing an iterative algorithm analysis on the test execution record to determine the local optimal weight corresponding to each dimensional indicator in the test execution record;

[0008] The benchmark performance of the distributed cluster is compared and analyzed from the multiple dimensional indicators based on the local optimal weight.

[0009] In one embodiment, before the step of executing the benchmark test in the test case, the step further includes:

[0010] Connecting to a stress testing client, where the stress testing client is used to perform benchmark testing;

[0011] In the case where it is determined that the stress testing client is installed with a benchmark testing tool for executing the benchmark test, by comparing the resource usage rate with a preset usage threshold, determining whether the stress testing client has a benchmark testing environment for executing the benchmark test;

[0012] In a case where the resource usage rates all meet the preset usage threshold, the step of performing the benchmark test based on the test case is performed.

[0013] In one embodiment, after the step of obtaining the test execution record of the test case, the step further includes:

[0014] When an abnormality occurs in the distributed cluster, the benchmark test is terminated and a corresponding abnormality log is generated;

[0015] When the distributed cluster is normal, determining whether the benchmark test under the test case reaches a performance bottleneck;

[0016] If a performance bottleneck is reached, the benchmark test of the test case is terminated, the benchmark test of the next test case is executed, and the current concurrency parameter is updated to the preset concurrency parameter;

[0017] If the performance bottleneck has not been reached, the preset concurrency parameters are adjusted and the benchmark test of the test case is continued.

[0018] In one embodiment, the step of determining whether the benchmark test under the test case has reached a performance bottleneck includes:

[0019] Calculate the request latency and CPU resource usage of the test case;

[0020] Determine whether the request delay reaches a preset delay threshold and whether the CPU resource usage reaches a preset resource threshold;

[0021] If the request delay reaches a preset delay threshold or the CPU resource usage reaches a preset resource threshold, it is determined that the benchmark test under the test case has reached a performance bottleneck.

[0022] In one embodiment, the step of performing an iterative algorithm analysis on the test execution record to determine the local optimal weight corresponding to each dimensional indicator in the test execution record includes:

[0023] Calculate the conditional expectation of each dimensional indicator in the test execution record according to the test execution record and preset initialization parameters;

[0024] Determine a maximization parameter of the conditional expectation, feed the maximization parameter back to the calculation process of the corresponding conditional expectation, and record the change of the maximization parameter, wherein the maximization parameter is used to maximize the conditional expectation;

[0025] When the change amount of the maximization parameter is less than a preset change threshold, the maximization parameter is determined to be the local optimal weight of the corresponding dimensional indicator.

[0026] In one embodiment, the dimensional indicators include operations per second, average latency, request latency, input traffic, and output traffic, and the step of comparing and analyzing the benchmark performance of the distributed cluster from the multiple dimensional indicators based on the local optimal weight includes:

[0027] Analyze the number of operations per second, average latency, request latency, input traffic, and output traffic under different test cases, and calculate the change range of the number of operations per second, average latency, request latency, input traffic, and output traffic based on the local optimal weights corresponding to the indicators in each dimension;

[0028] Integrate the variation range of the number of operations per second, the variation range of the average delay, the variation range of the request delay, the variation range of the input flow, and the variation range of the output flow into test analysis indicators;

[0029] If the test analysis index is greater than 0, it indicates that the baseline performance has improved;

[0030] If the test analysis index is less than 0, it indicates that the benchmark performance has degraded.

[0031] In addition, to achieve the above purpose, the present application also proposes a benchmark performance testing device, which is applied to a distributed cluster. The benchmark performance testing device includes:

[0032] A case test module, used to generate a test case according to the received case factors, and for any of the test cases, execute a benchmark test in the test case according to preset concurrency parameters to obtain a test execution record of the test case;

[0033] A weight determination module, used to perform an iterative algorithm analysis on the test execution record to determine the local optimal weight corresponding to each dimensional indicator in the test execution record;

[0034] A performance analysis module is used to compare and analyze the benchmark performance of the distributed cluster from multiple dimensional indicators based on the local optimal weight.

[0035] In addition, to achieve the above objectives, the present application also proposes an electronic device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the benchmark performance testing method described above.

[0036] In addition, to achieve the above objectives, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the benchmark performance testing method described above are implemented.

[0037] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the benchmark performance testing method described above are implemented.

[0038] The present application provides a benchmark performance testing method, which is applied to a distributed cluster. The benchmark performance testing method includes: generating a test case based on received case factors, and for any test case, executing the benchmark test in the test case according to preset concurrency parameters to obtain a test execution record of the test case; performing an iterative algorithm analysis on the test execution record to determine the local optimal weight corresponding to each dimensional indicator in the test execution record; and comparing and analyzing the benchmark performance of the distributed cluster from multiple dimensional indicators based on the local optimal weights.

[0039] This application solves the problem of single test cases and incomplete coverage in traditional testing methods through automated case design, making the test more comprehensive. By executing the benchmark test in the test case and recording the test execution results, the high concurrent requests of the distributed cluster in the real environment are simulated, and more accurate and reliable test data are obtained, which provides strong support for performance optimization. The local optimal weights of each dimensional indicator are automatically determined through an iterative algorithm, and higher weights are assigned to the dimensional indicators that have a greater impact on the evaluation benchmark performance, so that the performance evaluation results are more in line with the actual usage, and the objective and scientific allocation of weights is achieved, which improves the accuracy of performance evaluation. Through multi-dimensional and comprehensive analysis, the performance of the distributed cluster is comprehensively evaluated, and a detailed reference basis is provided for performance optimization. Compared with the manual design of test cases in related schemes, the case execution, test result collection and result comparison analysis are repeated through the command line. This application realizes accurate and efficient evaluation of the benchmark performance of the distributed cluster through the steps of automatically generating test cases, presetting concurrent parameters to execute tests, iterative algorithm analysis to determine weights, and multi-dimensional comprehensive evaluation, which effectively solves the problem of low test efficiency and provides strong support for the performance optimization of the distributed cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 A flowchart of the first embodiment of the benchmark performance testing method of the present application is provided;

[0043] Figure 2 A flowchart diagram of the second embodiment of the benchmark performance testing method of the present application is provided;

[0044] Figure 3 Automated case design and analysis flow chart for the benchmark performance testing method for this application;

[0045] Figure 4 A flowchart of intelligent result analysis of the benchmark performance testing method of this application;

[0046] Figure 5 This is a schematic diagram of the module structure of the benchmark performance testing device according to an embodiment of the present application;

[0047] Figure 6 Schematic diagram of the device structure of the hardware operating environment involved in the benchmark performance testing method in the embodiment of the present application.

[0048] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0049] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0050] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0051] This application is applied to distributed clusters, and the main solution is: generate test cases based on the received case factors, and for any test case, execute the benchmark test in the test case according to the preset concurrency parameters to obtain the test execution record of the test case; perform iterative algorithm analysis on the test execution record to determine the local optimal weight corresponding to each dimensional indicator in the test execution record; and compare and analyze the benchmark performance of the distributed cluster from multiple dimensional indicators based on the local optimal weights.

[0052] In this embodiment, for ease of description, the following description is made with the benchmark performance test system as the execution subject.

[0053] The main shortcoming of benchmark performance testing in current distributed clusters is the lack of automation. Testers need to manually design cases, log in to servers, execute tests and analyze results. This process is cumbersome and inefficient, resulting in a lot of time and manpower waste. In addition, the testing process lacks intelligent analysis, making it difficult to quickly and accurately locate performance bottlenecks and optimization directions.

[0054] This application provides a solution, which executes benchmark tests and records test results through automated case design, and uses an iterative algorithm to automatically determine the weights of indicators in each dimension, so as to evaluate the performance of distributed clusters in a multi-dimensional and comprehensive manner, simulate high-concurrency requests in a real environment, ensure the comprehensiveness of the test, and provide accurate and reliable test data, thereby achieving accurate and efficient evaluation of the benchmark performance of distributed clusters, effectively solving the problem of low test efficiency, and improving test efficiency.

[0055] It should be noted that the execution subject of this embodiment may be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a benchmark performance test system, etc. The following takes the benchmark performance test system as an example to illustrate this embodiment and the following embodiments.

[0056] Based on this, the present application embodiment provides a benchmark performance testing method, which is applied to a distributed cluster. Figure 1 , Figure 1 This is a flowchart of the first embodiment of the benchmark performance testing method of the present application.

[0057] In this embodiment, the benchmark performance testing method includes steps S01 to S03:

[0058] Step S01, generating a test case according to the received case factors, and for any test case, executing a benchmark test in the test case according to preset concurrency parameters to obtain a test execution record of the test case;

[0059] It should be noted that a distributed cluster refers to a network composed of multiple computers that work together to provide high-performance computing and storage services. Taking the distributed cluster as an example, distributed clusters are often used to improve data availability and scalability. Case factors refer to parameters or conditions used to generate test cases, including test scenarios, data types, operation types, etc. (for example, when comparing different architecture types, cluster architecture and master-slave architecture are a pair of case factors). These factors determine the specific content and test objectives of the test cases. Test cases are specific test scenarios generated based on case factors, which are used to simulate operations in actual use. Each test case contains a set of specific test conditions and test steps. Preset concurrency parameters are parameters such as the number of concurrent users and request rates set before the test, which are used to simulate performance under different loads. These parameters determine the load and stress level of the test case. Benchmark testing is to perform performance testing on the system under specific conditions to evaluate its performance level, and is used to compare performance differences between different systems or different configurations. Test execution records are used to record various data during the execution of test cases, such as OPS, average latency, P99 latency, etc. These data are the basis for subsequent analysis and evaluation.

[0060] In addition, it should be noted that the user analyzes and fills in the case factors according to the test purpose. These case factors can be various variables that affect the performance of the Redis service, such as cluster architecture (cluster architecture and master-slave architecture as a comparison), data type, operation type (read, write, delete, etc.), data size, etc. After the system receives these case factors, it automatically generates all possible test case combinations based on the existing permutation and combination algorithm. Assuming there are n pairs of case factors, a total of 2 can be combined n For each generated test case, the user needs to fill in the relevant information of the Redis cluster, including the IP address, password, port number, etc. of each node in the cluster, and set the initial test concurrency parameters. If similar benchmark tests have been conducted before and the optimal concurrency value has been determined (the number of concurrent client connections that can enable the distributed cluster server to achieve the highest throughput or lowest latency under a specific system environment and resource configuration), the user can directly use these values ​​to reduce the test preparation time. When the system is tested for the first time, the user needs to manually configure the initial concurrency value. After that, the program will automatically complete the benchmark execution of all cases and generate test execution records.

[0061] It is understandable that in traditional testing methods, the design of test cases often relies on the experience and intuition of testers, which may result in test cases that are not comprehensive enough or have omissions. In addition, manually configuring the test environment and executing the test process are cumbersome and time-consuming, which reduces the test efficiency. Therefore, step S01 is performed. By automatically generating test cases and executing benchmark tests, the system can cover more test scenarios and load conditions, thereby more accurately evaluating the benchmark performance of the distributed cluster. At the same time, the automated testing process reduces manual intervention and improves the stability and repeatability of the test.

[0062] Step S02, performing an iterative algorithm analysis on the test execution record to determine the local optimal weight corresponding to each dimensional indicator in the test execution record;

[0063] It should be noted that the iterative algorithm can automatically adjust the weight distribution to optimize the evaluation results. The iterative algorithm analysis uses the iterative algorithm to conduct an in-depth analysis of the test execution records to determine the local optimal weights of each dimensional indicator. Dimensional indicators are used to evaluate various aspects of system performance, such as OPS, average latency, P99 latency, input traffic, output traffic, etc. At the same time, other dimensional indicators can also be configured according to demand to analyze different performance characteristics of the system. The local optimal weight refers to the optimal weight distribution of each dimensional indicator in the performance evaluation under a specific test scenario, which reflects the importance of different dimensional indicators in the overall performance evaluation.

[0064] In addition, it should be noted that after obtaining the test execution records, an iterative algorithm (such as the EM clustering algorithm, the particle swarm optimization algorithm, etc.) is used to conduct an in-depth analysis of these records to determine the local optimal weights of each dimensional indicator when evaluating the distributed cluster benchmark performance.

[0065] It is understandable that in traditional evaluation methods, the weights of indicators in each dimension are often determined subjectively by testers, resulting in the evaluation results being not objective and accurate enough. At the same time, with the development of distributed clusters and changes in application scenarios, the importance of indicators in each dimension may also change. Therefore, step S02 is performed to automatically adjust and determine the local optimal weights of indicators in each dimension through iterative algorithm analysis, so that the performance evaluation results are more in line with actual usage, thereby improving the accuracy and credibility of the evaluation and providing strong data support for the optimization and upgrade of distributed clusters.

[0066] Step S03, comparing and analyzing the benchmark performance of the distributed cluster from multiple dimensional indicators based on the local optimal weight.

[0067] It should be noted that after determining the local optimal weights of indicators in each dimension, the system will conduct a comprehensive analysis of the test results of multiple test cases based on these weights. By comparing the performance of different test cases under the same or different weights, the benchmark performance of the distributed cluster can be comprehensively evaluated. Benchmark performance refers to the performance level of the system under specific conditions. It is an important reference for evaluating system performance, used to compare performance differences between different systems or different configurations, and provide guidance for system optimization.

[0068] It is understandable that due to unreasonable weight distribution or single evaluation indicators in traditional evaluation methods, the evaluation results may not be comprehensive and accurate. Therefore, step S03 is performed to compare the performance of different test cases under the same or different weights, comprehensively evaluate the benchmark performance of the distributed cluster, and provide a detailed reference basis for the optimization and upgrade of the distributed cluster, which helps to improve the performance and stability of the service.

[0069] In a feasible implementation manner, in step S01, before the step of executing the benchmark test in the test case, steps A01 to A03 are also included:

[0070] Step A01, connecting to a stress testing client, where the stress testing client is used to perform benchmark testing;

[0071] It should be noted that the stress testing client refers to the remote computer or server used to perform benchmark testing. The system establishes communication with the stress testing client through a network connection to ensure that commands can be executed remotely and data can be transmitted.

[0072] Step A02, when it is determined that the stress testing client has installed a benchmark testing tool for executing the benchmark test, by comparing the resource usage rate with a preset usage threshold, determining whether the stress testing client has a benchmark testing environment for executing the benchmark test;

[0073] It should be noted that a specific command (such as redis-benchmark --version or a command to check the software list installed on the system) is executed on the server (used to provide distributed cluster services, process requests from the stress test client, and return results), and the response content R of the stress test client is captured to check whether the response content R contains the preset keyword M. i(such as redis-benchmark, benchmark, etc.), the preset keyword refers to the key string used to determine whether the stress test client has installed the benchmark tool, such as redis-benchmark, benchmark, etc. As long as any preset keyword is included, the stress test client is evaluated as "qualified", indicating that the stress test client has installed the benchmark tool. The benchmark tool refers to a software tool used to perform distributed cluster benchmark tests, such as redis-benchmark. In addition, multiple groups of keyword parameters can be set for conditional evaluation (here it is assumed that 3 groups are set). The evaluation expression for determining whether the benchmark tool is installed is:

[0074] Result first =M1∈R||M2∈R||M3∈R

[0075] In addition, it should be noted that resource utilization refers to the usage of CPU, memory, disk and other resources required by the server when executing the benchmark test. The preset usage threshold refers to the upper limit of resource utilization that is pre-set according to the requirements of the distributed cluster benchmark test and the hardware configuration of the stress test client. The benchmark test environment refers to the sum of all conditions and resources required to execute the distributed cluster benchmark test, including test tools, hardware resources, etc. After confirming that the benchmark test tool is installed on the stress test client, the system further obtains the server's CPU utilization, memory utilization, disk space remaining and other resource utilization, and compares them with the preset usage threshold. If each resource utilization meets the corresponding preset usage threshold, the current server is evaluated as "qualified", indicating that the current server has the benchmark test execution environment.

[0076] For example, assuming that the CPU usage threshold is avg CPU 、Memory usage threshold avg Memory , Disk Disk space remaining threshold is avg Disk , the evaluation expression for judging whether the benchmark execution environment is available is:

[0077]

[0078] Step A03, when the resource utilization rates all meet the preset threshold values, a step of performing a benchmark test based on the test case is executed.

[0079] It should be noted that after confirming that the stress testing client has the benchmark tools and benchmark execution environment for executing benchmark tests, the benchmark commands are spliced ​​out according to the distributed cluster related information filled in by the user, and the benchmark commands are executed in a loop on the stress testing client. The benchmark commands are usually one or more command line instructions used to start the benchmark tool and specify the test parameters.

[0080] In this implementation, by establishing a communication bridge with the stress testing client, the necessary connection conditions are provided for subsequent operations. Through keyword matching, it is quickly determined whether the stress testing client has the benchmark testing tool to perform the benchmark test. The conditional evaluation of multiple groups of keyword parameters improves the accuracy and flexibility of the judgment. By verifying the resource utilization rate, it is ensured that the test environment has sufficient resources to support the execution of the benchmark test, thereby improving the accuracy and efficiency of the test.

[0081] In a feasible implementation manner, in step S01, after the step of obtaining the test execution record of the test case, steps A11 to A14 are also included:

[0082] Step A11, when there is an abnormality in the distributed cluster, terminate the benchmark test and generate a corresponding abnormality log;

[0083] It should be noted that the exception log refers to a log file that is automatically generated during the test and contains exception information, timestamps, possible causes, etc. when the system detects an exception in the distributed cluster. During the distributed cluster benchmark test, the system will monitor the cluster status in real time and judge the distributed cluster status by comparing the buffer size and the number of connected slave nodes. If any item does not meet the condition threshold, the cluster is considered to be abnormal (such as the buffer size exceeds the preset threshold, the number of connected slave nodes is lower than the threshold, etc.). Once an abnormality is found in the cluster, the currently executing benchmark test will be terminated immediately, and the corresponding exception log will be generated for the tester to investigate the cause of the abnormality. Assume that the buffer threshold is max buffer , the threshold for the number of connected slave nodes is max slave , the evaluation expression for judging whether the distributed cluster is abnormal is:

[0084] Cluster=!(Bufferi≥max buffer ||Slave i ≠max slave )

[0085] Step A12, when the distributed cluster is normal, determine whether the benchmark test under the test case reaches a performance bottleneck;

[0086] It should be noted that when the distributed cluster is running normally, the system will determine whether the benchmark test under the current test case has reached a performance bottleneck. Performance bottleneck refers to the state in which the system cannot continue to improve performance due to reaching the limit of one or some key indicators in the distributed cluster benchmark performance test. These key indicators include P99 latency (99% request response time), CPU resource utilization, etc. For example, when the P99 latency reaches the preset threshold (such as 500 milliseconds), it can be considered that the system has reached a performance bottleneck.

[0087] Step A13: if a performance bottleneck is reached, the benchmark test of the test case is terminated, the benchmark test of the next test case is executed, and the current concurrency parameter is updated to the preset concurrency parameter;

[0088] It should be noted that during the benchmark test, if the system detects that the current test case has reached a performance bottleneck, it will terminate the benchmark test of the test case and jump directly to the next test case. At the same time, it will update the current concurrency parameters to the preset concurrency parameters, indicating the current optimal concurrency parameters, in preparation for the test of the next case.

[0089] Step A14: If the performance bottleneck has not been reached, the preset concurrency parameters are adjusted and the benchmark test of the test case is continued.

[0090] It should be noted that if the current test case has not reached the performance bottleneck, the preset concurrency parameters are adjusted to gradually approach the performance limit of the system, so as to more accurately evaluate the performance of the Redis service, and the benchmark test of the test case is continued using the adjusted preset concurrency parameters.

[0091] In this implementation, the accuracy and reliability of the test are ensured through real-time monitoring and exception handling mechanisms. At the same time, the generation of exception logs also provides testers with clues to investigate the causes of exceptions. By setting reasonable performance bottleneck judgment conditions, it is possible to accurately determine whether the system has reached its performance limit, thereby ensuring the accuracy and effectiveness of the test results and providing strong data support for subsequent performance optimization. By introducing a dynamic concurrency parameter adjustment mechanism, concurrency parameters are automatically adjusted and continuous testing is performed, thereby achieving more comprehensive performance testing and ensuring the comprehensiveness and accuracy of the test results.

[0092] In a feasible implementation manner, in step A12, the step of determining whether the benchmark test under the test case has reached a performance bottleneck includes steps A21 to A23:

[0093] Step A21, calculating the request delay and CPU resource usage of the test case;

[0094] It should be noted that the system collects and calculates performance data for the currently executed test cases, including request latency and CPU resource utilization. Request latency (such as P99 latency) refers to the time it takes from the client sending a request to receiving a response, which reflects the speed at which the distributed cluster processes requests. CPU resource utilization refers to the proportion of CPU resources occupied by the distributed cluster when processing these requests, which reflects the computing efficiency of the distributed cluster.

[0095] Step A22, determining whether the request delay reaches a preset delay threshold and whether the CPU resource usage reaches a preset resource threshold;

[0096] It should be noted that the system compares the calculated request delay and CPU resource utilization with the preset thresholds. The preset delay threshold and preset resource threshold are set according to the performance requirements and expectations of the distributed cluster, and they reflect the system's expected standards for performance and resource utilization.

[0097] Step A23: If the request delay reaches a preset delay threshold or the CPU resource usage reaches a preset resource threshold, it is determined that the benchmark test under the test case has reached a performance bottleneck.

[0098] It should be noted that, according to the judgment result of step A22, if any performance data reaches or exceeds the preset threshold (the request delay reaches the preset delay threshold or the CPU resource utilization reaches the preset resource threshold), it is determined that the benchmark test under the current test case has reached a performance bottleneck, which means that under the current test conditions, the performance of the distributed cluster can no longer meet the expected standards, and further adjustment of test parameters or optimization of service performance is required.

[0099] In this embodiment, by accurately calculating the request delay and CPU resource utilization, the system can more accurately evaluate the performance of the distributed cluster and provide reliable data support for subsequent performance bottleneck judgments. By comparing the request delay and CPU resource utilization with the corresponding preset thresholds, the system can timely and accurately determine whether the distributed cluster has reached a performance bottleneck, thereby avoiding invalid testing and waste of resources. By promptly terminating test cases that have reached performance bottlenecks, the system can more effectively utilize test resources, improve test efficiency, and provide accurate data support for subsequent performance optimization.

[0100] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 2 In step S02, the test execution record is analyzed by an iterative algorithm, and the step of determining the local optimal weight corresponding to each dimensional indicator in the test execution record includes steps B01 to B03:

[0101] Step B01, calculating the conditional expectation of each dimension indicator in the test execution record according to the test execution record and preset initialization parameters;

[0102] It should be noted that the weight factors of each dimension are calculated by the EM clustering algorithm. The EM algorithm is an iterative algorithm used to solve the parameters of the probability distribution model containing latent variables. It is derived using the maximum likelihood criterion, which holds that for given observed data, the parameter values ​​of the model should maximize the probability of the observed data. The EM algorithm gradually optimizes the model parameters through an iterative method until convergence. It consists of two steps: the E step and the M step.

[0103] In addition, it should be noted that the system obtains test execution records, which include performance indicators of multiple dimensions, such as operations per second (OPS), average latency, P99 latency, input and output traffic, etc. At the same time, the system will also use preset initialization parameters. These parameters are set based on historical data or empirical values. They are used to guide the calculation of conditional expectations, namely the E step in the EM algorithm. Conditional expectations refer to the values ​​that are expected to be achieved by indicators of each dimension under a given set of parameters (i.e., preset initialization parameters) and observed data (i.e., test execution records).

[0104] For example, assuming x i is the test execution record (including OPS, average delay, P99 delay, input and output traffic), i is a latent variable, and the parameter θ represents the parameter of the model. For each x i , conditional expectation Q i The calculation formula is:

[0105] Q i (z i )=p(z i |x i ;θ)

[0106] Step B02, determining a maximization parameter of the conditional expectation, and feeding the maximization parameter back to the calculation process of the corresponding conditional expectation, recording the change of the maximization parameter, and the maximization parameter is used to maximize the conditional expectation;

[0107] It should be noted that the system will try to find a set of parameters (i.e., maximization parameters) so that the conditional expectation of indicators in each dimension is maximized under given observation data. This is usually achieved through an iterative optimization algorithm, such as the M step in the EM algorithm. The maximization parameter refers to the parameter value that can maximize the conditional expectation. In each iteration, the system will feed back the currently found maximization parameter to the calculation process of the conditional expectation (such as the E step in the EM algorithm) and record the change in the maximization parameter to determine whether the iteration has converged.

[0108] For example, the conditional expectation Q obtained in step B01 is i To calculate the maximized parameter θ, the maximized parameter will be used in the next iterative calculation (i.e., the E step in the EM algorithm). This process is repeated continuously. The calculation formula for the maximized parameter θ is:

[0109]

[0110] Step B03: When the change of the maximization parameter is less than the preset change threshold, the maximization parameter is determined to be the local optimal weight of the corresponding dimension indicator.

[0111] It should be noted that, in order to judge whether the change of the maximization parameter is less than the preset change threshold, the preset change threshold refers to the threshold used to judge whether the change of the maximization parameter is small enough to consider the iteration convergence. If it is less than the preset change threshold, the iteration is considered to have converged. The current maximization parameter is the local optimal weight of the corresponding dimensional indicator. The local optimal weight refers to the parameter configuration that can make the conditional expectation of each dimensional indicator reach the local maximum under the current observation data and parameter space.

[0112] In this implementation, the conditional expectation is calculated to more accurately understand the performance of indicators in each dimension under specific conditions, which provides a basis for subsequent optimization. By maximizing the conditional expectation, a set of optimal parameter configurations can be found to achieve the best performance of the distributed cluster. By judging whether the iteration converges, it is ensured that the local optimal weights found are stable and reliable, which is helpful for subsequent performance optimization and decision making.

[0113] In a feasible implementation manner, the dimensional indicators include operations per second, average latency, request latency, input traffic, and output traffic. In step S03, the step of comparing and analyzing the benchmark performance of the distributed cluster from multiple dimensional indicators based on the local optimal weight includes steps B21 to B24:

[0114] Step B21, analyzing the number of operations per second, average delay, request delay, input traffic, and output traffic under different test cases, and calculating the change range of the number of operations per second, the change range of the average delay, the change range of the request delay, the change range of the input traffic, and the change range of the output traffic respectively based on the local optimal weights corresponding to the indicators of each dimension;

[0115] It should be noted that the number of operations per second (OPS) refers to the number of operations that the distributed cluster can process per second, the average latency refers to the average response time of the distributed cluster to process requests, the request latency refers to the latency at a specific percentile (such as P99), which is used to measure the response time in extreme cases, the input traffic refers to the data traffic received by the distributed cluster, and the output traffic refers to the data traffic sent by the distributed cluster. The change range of the number of operations per second, the average latency, the request latency, the input traffic, and the output traffic respectively represent the change amount of each dimensional indicator under different test cases.

[0116] In addition, it should be noted that the system analyzes the number of operations per second (OPS), average latency, request latency, input traffic, and output traffic of the distributed cluster under different test cases, and calculates the change range of the number of operations per second, the change range of the average latency, the change range of the request latency, the change range of the input traffic, and the change range of the output traffic based on the local optimal weights corresponding to the indicators of each dimension.

[0117] For example, the number of operations per second (OPS) varies by 0. i , the corresponding local optimal weight is According to the average number of operations per second avg under different cases 案例iops , the calculation formula for the change in the number of operations per second is:

[0118]

[0119] The average delay variation is R i , the corresponding local optimal weight is According to the average delay avg under different cases 案例i平均延时 , the calculation formula for the change range of average delay is:

[0120]

[0121] The request delay variation is R99 i , the corresponding local optimal weight is According to the request delay avg in different cases 案例iP99延时 , the calculation formula for the change in request delay is:

[0122]

[0123] The input flow rate changes to IN i , the corresponding local optimal weight is According to the input flow avg under different cases 案例i输入流量 , the calculation formula for the change range of input flow is:

[0124]

[0125] Output flow rate change range is OUT i , the corresponding local optimal weight is According to the output flow avg under different cases 案例i输出流量 , the calculation formula for the change range of output flow is:

[0126]

[0127] Step B22, integrating the change range of the number of operations per second, the change range of the average delay, the change range of the request delay, the change range of the input flow and the change range of the output flow into test analysis indicators;

[0128] It should be noted that the system performs weighted summation or weighted average operations on the change ranges of the indicators of each dimension calculated in step B21 to obtain an indicator that comprehensively reflects the performance change, which is recorded as the test analysis indicator.

[0129] Step B23, if the test analysis index is greater than 0, it indicates that the baseline performance is improved;

[0130] It should be noted that the system determines whether the performance of the distributed cluster has improved or declined based on the size of the test analysis indicator. If the test analysis indicator is greater than 0, it means that the performance has improved.

[0131] Step B24: If the test analysis index is less than 0, it indicates that the benchmark performance has degraded.

[0132] It should be noted that the system determines whether the performance of the distributed cluster has improved or declined based on the size of the test analysis indicator. If it is less than 0, it means that the performance has declined.

[0133] In this implementation, by introducing weights, the test results can better reflect the actual performance changes of the distributed cluster, improve the accuracy and effectiveness of the test, and provide basic data for subsequent test analysis. By integrating test analysis indicators, the system can provide users with a concise and clear test result, which is convenient for users to quickly understand the performance changes of the distributed cluster. Through clear judgment criteria, the system can provide users with an accurate test result, which is convenient for users to take corresponding optimization measures according to the test results, thereby improving the performance of the distributed cluster.

[0134] For example, to help understand the technical concept or technical principle of this application, please refer to Figure 3 and Figure 4 , Figure 3 Provides automated case design and analysis flow chart, Figure 4 Provides a flow chart for intelligent result analysis.

[0135] exist Figure 3 In the test case generation process, the test case is generated according to the combination of case factors filled in by the user. After filling in the test information and setting the experience gradient concurrency parameters for each case, it is determined whether the system has the conditions for test execution. If the system has the conditions for test execution, the test command is executed in a loop. After each round of test execution, the stress test data on the stress test server is collected. At the same time, the monitoring data of the distributed cluster in the corresponding time period is collected to determine whether the cluster has any abnormalities. If there are no abnormalities, it continues to determine whether the performance bottleneck is reached. If the cluster has an abnormality or reaches a performance bottleneck, the stress test program is stopped, the corresponding judgment results and test execution records are output, and intelligent result analysis is performed. If there are no abnormalities and the performance bottleneck has not been reached, the concurrency parameters are adjusted and the benchmark test is executed.

[0136] Figure 4 In Figure 3The specific steps of intelligent result analysis are as follows: after obtaining the test execution record, perform EM analysis, set the initialization parameters and calculate the conditional expectation in the E step, estimate the maximization parameters in the M step, and continuously cycle the E step and M step until the iteration converges and determines the local optimal weight. Based on the local optimal weight, calculate the OPS, average delay, P99 delay, input traffic, output traffic and other dimensions for comparative analysis results, and output the test execution record and comparative analysis results.

[0137] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the benchmark performance testing method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0138] This application also provides a benchmark performance testing device, which is applied to distributed clusters. Please refer to Figure 5 , the benchmark performance test setup includes:

[0139] The case test module 10 is used to generate a test case according to the received case factors, and for any test case, execute the benchmark test in the test case according to the preset concurrency parameters to obtain the test execution record of the test case;

[0140] The weight determination module 20 is used to perform an iterative algorithm analysis on the test execution record to determine the local optimal weight corresponding to each dimensional indicator in the test execution record;

[0141] The performance analysis module 30 is used to compare and analyze the benchmark performance of the distributed cluster from multiple dimensional indicators based on the local optimal weight.

[0142] Optionally, the benchmark performance testing device further includes a test judgment module 40, and the test judgment module 40 is used to:

[0143] Connect to the stress testing client, which is used to perform benchmark tests;

[0144] In the case where it is determined that the stress testing client is installed with a benchmark testing tool for executing the benchmark test, by comparing the resource usage rate with a preset usage threshold, it is determined whether the stress testing client has a benchmark testing environment for executing the benchmark test;

[0145] When the resource usage rates all meet the preset usage thresholds, a step of performing a benchmark test based on the test case is performed.

[0146] Optionally, the benchmark performance testing device further includes a bottleneck determination module 50, and the bottleneck determination module 50 is used to:

[0147] If there is an exception in the distributed cluster, the benchmark test is terminated and the corresponding exception log is generated;

[0148] When the distributed cluster is normal, determine whether the benchmark under the test case has reached a performance bottleneck;

[0149] If a performance bottleneck is reached, the benchmark test of the test case is terminated, the benchmark test of the next test case is executed, and the current concurrency parameter is updated to the preset concurrency parameter;

[0150] If the performance bottleneck is not reached, adjust the preset concurrency parameters and continue to execute the benchmark test case.

[0151] Optionally, the bottleneck determination module 50 is further configured to:

[0152] Calculate the request latency and CPU resource usage of the test case;

[0153] Determine whether the request delay reaches the preset delay threshold and whether the CPU resource usage reaches the preset resource threshold;

[0154] If the request delay reaches a preset delay threshold or the CPU resource usage reaches a preset resource threshold, it is determined that the benchmark test under the test case has reached a performance bottleneck.

[0155] Optionally, the weight determination module 20 is further configured to:

[0156] Calculate the conditional expectation of each dimension indicator in the test execution record according to the test execution record and the preset initialization parameters;

[0157] Determine the maximization parameter of the conditional expectation, and feed the maximization parameter back to the calculation process of the corresponding conditional expectation, record the change of the maximization parameter, and the maximization parameter is used to maximize the conditional expectation;

[0158] When the change in the maximization parameter is less than a preset change threshold, the maximization parameter is determined to be the local optimal weight of the corresponding dimension indicator.

[0159] Optionally, the dimensional indicators include the number of operations per second, average latency, request latency, input traffic, and output traffic. The performance analysis module 30 is further used to:

[0160] Analyze the number of operations per second, average latency, request latency, input traffic, and output traffic under different test cases, and calculate the change range of the number of operations per second, average latency, request latency, input traffic, and output traffic based on the local optimal weights corresponding to the indicators in each dimension;

[0161] Integrate the change range of operations per second, average latency, request latency, input traffic, and output traffic into test analysis indicators;

[0162] If the test analysis index is greater than 0, it means that the baseline performance has improved;

[0163] If the test analysis index is less than 0, it means that the benchmark performance has degraded.

[0164] The benchmark performance testing device provided by the present application adopts the benchmark performance testing method in the above embodiment, which can solve the technical problem of low test efficiency. Compared with the prior art, the beneficial effects of the benchmark performance testing device provided by the present application are the same as the beneficial effects of the benchmark performance testing method provided by the above embodiment, and other technical features in the benchmark performance testing device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0165] The present application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the benchmark performance testing method in the above-mentioned embodiment one.

[0166] Reference below Figure 6 , which shows a schematic diagram of the structure of an electronic device suitable for implementing an embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, notebook computers, PADs (Portable Application Description: tablet computers), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0167] like Figure 6As shown, the electronic device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the electronic device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows an electronic device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0168] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0169] The electronic device provided by the present application adopts the benchmark performance test method in the above embodiment to solve the technical problem of low test efficiency. Compared with the prior art, the beneficial effects of the electronic device provided by the present application are the same as the beneficial effects of the benchmark performance test method provided by the above embodiment, and the other technical features in the electronic device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0170] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0171] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0172] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the benchmark performance testing method in the above-mentioned embodiment.

[0173] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0174] The computer-readable storage medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

[0175] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by an electronic device, the benchmark performance testing device: generates a test case according to the received case factors, and for any test case, executes the benchmark test in the test case according to the preset concurrency parameters to obtain a test execution record of the test case; performs an iterative algorithm analysis on the test execution record to determine the local optimal weight corresponding to each dimensional indicator in the test execution record; and compares and analyzes the benchmark performance of the distributed cluster from multiple dimensional indicators based on the local optimal weights.

[0176] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0177] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0178] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0179] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned benchmark performance testing method, and can solve the technical problem of low test efficiency. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the benchmark performance testing method provided by the above-mentioned embodiment, and will not be described in detail here.

[0180] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned benchmark performance testing method when executed by a processor.

[0181] The computer program product provided by the present application can solve the technical problem of low test efficiency. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the benchmark performance testing method provided by the above embodiment, which will not be repeated here.

[0182] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A benchmark performance testing method, characterized in that: Applied to distributed clusters, the benchmark performance testing method includes: Generate a test case according to the received case factors, and for any of the test cases, execute a benchmark test in the test case according to preset concurrency parameters to obtain a test execution record of the test case; Performing an iterative algorithm analysis on the test execution record to determine the local optimal weight corresponding to each dimensional indicator in the test execution record; The benchmark performance of the distributed cluster is compared and analyzed from the multiple dimensional indicators based on the local optimal weight.

2. The benchmark performance testing method according to claim 1, characterized in that: The step of executing the benchmark test in the test case also includes: Connecting to a stress testing client, where the stress testing client is used to perform benchmark testing; In the case where it is determined that the stress testing client is installed with a benchmark testing tool for executing the benchmark test, by comparing the resource usage rate with a preset usage threshold, determining whether the stress testing client has a benchmark testing environment for executing the benchmark test; In a case where the resource usage rates all meet the preset usage threshold, the step of performing the benchmark test based on the test case is performed.

3. The benchmark performance testing method according to claim 1, characterized in that: After the step of obtaining the test execution record of the test case, the method further includes: When an abnormality occurs in the distributed cluster, the benchmark test is terminated and a corresponding abnormality log is generated; When the distributed cluster is normal, determining whether the benchmark test under the test case reaches a performance bottleneck; If a performance bottleneck is reached, the benchmark test of the test case is terminated, the benchmark test of the next test case is executed, and the current concurrency parameter is updated to the preset concurrency parameter; If the performance bottleneck has not been reached, the preset concurrency parameters are adjusted and the benchmark test of the test case is continued.

4. The benchmark performance testing method according to claim 3, characterized in that: The step of determining whether the benchmark test under the test case has reached a performance bottleneck comprises: Calculate the request latency and CPU resource usage of the test case; Determine whether the request delay reaches a preset delay threshold and whether the CPU resource usage reaches a preset resource threshold; If the request delay reaches a preset delay threshold or the CPU resource usage reaches a preset resource threshold, it is determined that the benchmark test under the test case has reached a performance bottleneck.

5. The benchmark performance testing method according to claim 1, characterized in that: The step of performing iterative algorithm analysis on the test execution record to determine the local optimal weight corresponding to each dimensional indicator in the test execution record includes: Calculate the conditional expectation of each dimensional indicator in the test execution record according to the test execution record and preset initialization parameters; Determine a maximization parameter of the conditional expectation, feed the maximization parameter back to the calculation process of the corresponding conditional expectation, and record the change of the maximization parameter, wherein the maximization parameter is used to maximize the conditional expectation; When the change amount of the maximization parameter is less than a preset change threshold, the maximization parameter is determined to be the local optimal weight of the corresponding dimensional indicator.

6. The benchmark performance testing method according to claim 1, wherein the dimension indicators include operations per second, average latency, request latency, input traffic, and output traffic, and wherein: The step of comparing and analyzing the benchmark performance of the distributed cluster from the multiple dimensional indicators based on the local optimal weight includes: Analyze the number of operations per second, average latency, request latency, input traffic, and output traffic under different test cases, and calculate the change range of the number of operations per second, average latency, request latency, input traffic, and output traffic based on the local optimal weights corresponding to the indicators in each dimension; Integrate the variation range of the number of operations per second, the variation range of the average delay, the variation range of the request delay, the variation range of the input flow, and the variation range of the output flow into test analysis indicators; If the test analysis index is greater than 0, it indicates that the baseline performance has improved; If the test analysis index is less than 0, it indicates that the benchmark performance has degraded.

7. A benchmark performance testing device, characterized in that: Applied to distributed clusters, the benchmark performance testing device includes: A case test module, used to generate a test case according to the received case factors, and for any of the test cases, execute a benchmark test in the test case according to preset concurrency parameters to obtain a test execution record of the test case; A weight determination module, used to perform an iterative algorithm analysis on the test execution record to determine the local optimal weight corresponding to each dimensional indicator in the test execution record; A performance analysis module is used to compare and analyze the benchmark performance of the distributed cluster from multiple dimensional indicators based on the local optimal weight.

8. An electronic device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the benchmark performance testing method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the benchmark performance testing method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the benchmark performance testing method according to any one of claims 1 to 6 are implemented.

Images