A multi-dimensional performance evaluation method and system for a computing power server

Abstract

This invention discloses a multi-dimensional performance evaluation method and system for computing servers, relating to the field of server performance data analysis. By constructing existing computing server resources into single server systems, server cluster systems, and distributed server systems, this invention selects the most suitable server system for running the test task based on its characteristics during performance testing. It then evaluates the performance of existing computing servers by selecting the optimal value from four dimensions: deployment time, running time, upgrade time, and uninstallation time. This achieves comprehensive performance testing and full utilization of existing resources, ensuring that the final evaluation results reflect the best performance achievable under existing computing server resource conditions, thus improving the accuracy of performance testing.

Description

Technical Field

[0001] This invention belongs to the field of server performance data analysis, and specifically relates to a multi-dimensional performance evaluation method and system for computing servers. Background Technology

[0002] Traditional methods do not consider the performance differences between single-machine / cluster / distributed architectures, and only use a single system form for system performance evaluation. This fails to fully utilize existing computing server resources, resulting in inaccurate evaluation results. In addition, traditional performance evaluation schemes require repeated testing and verification, and cannot quickly evaluate the adaptability performance of new tasks through historical feature clustering. Summary of the Invention

[0003] In response to the problems in related technologies, this invention proposes a multi-dimensional performance evaluation method and system for computing servers to overcome the aforementioned technical problems existing in the existing related technologies.

[0004] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution:

[0005] This invention provides a multi-dimensional performance evaluation method for computing servers, comprising the following steps:

[0006] S1. Collect data on factors that affect the performance of a single server system, a server cluster system, and a distributed server system, respectively.

[0007] S2. Collect feature data from multiple current model building tasks based on the single server system, server cluster system, and server distributed system in S1.

[0008] S3. Deploy, run, upgrade, and uninstall the multiple current model building tasks in S2 in the single server system, server cluster system, and server distributed system in S1, and record the corresponding time consumption data.

[0009] S4. Based on the factor data in S1, the feature data in S2, and the time consumption data recorded in S3, construct mapping models corresponding to the deployment, operation, upgrade, and uninstallation times, respectively.

[0010] S5. Input the feature data of multiple performance testing tasks and the factor data collected in S1 into the mapping model constructed in S4 for mapping.

[0011] S6. Calculate the cluster center of the feature data of the task corresponding to the mapping data that meets the preset time consumption threshold in S5.

[0012] S7. Calculate the Euclidean distance between the cluster centers of the feature data of the newly added test task and the feature data obtained in S6, and evaluate the performance of the current computing server based on the minimum Euclidean distance for deployment, operation, upgrade and uninstallation.

[0013] Preferably, step S1 includes the following steps:

[0014] S11. Based on the current total resources, set up several identical computing power servers to obtain the current computing power server set; based on the current computing power server set, construct a single server system, a server cluster system, and a server distributed system respectively;

[0015] S12. Define the types of factors that affect the performance of a single server system, a server cluster system, and a server distributed system, respectively, to obtain the first set of performance influencing factor types, the second set of performance influencing factor types, and the third set of performance influencing factor types.

[0016] In conjunction with the first set of performance influencing factor types, the second set of performance influencing factor types, and the third set of performance influencing factor types, performance influencing factor data for single server systems, server cluster systems, and server distributed systems are collected respectively to obtain the first performance influencing factor dataset, the second performance influencing factor dataset, and the third performance influencing factor dataset;

[0017] Since single computing servers, server clusters, and distributed servers have different application scopes, this solution prevents subsequent erroneous evaluations of the performance of existing computing servers. It deploys several existing computing servers in one of these three modes to allow for selection of the most suitable mode based on the type of task being run, ensuring full utilization of the existing computing server performance and guaranteeing the accuracy and comprehensiveness of the evaluation. By using a unified hardware set of computing servers, the interference of device heterogeneity on test results is eliminated, ensuring the comparability of performance data. All evaluations are based on the same fundamental computing unit, ensuring that the performance differences between single-machine, cluster, and distributed systems truly reflect the architectural characteristics.

[0018] Preferably, step S2 includes the following steps:

[0019] S21. Set several tasks to be run to obtain the current model building task set; in conjunction with a single server system, a server cluster system, and a server distributed system, set several feature types of the current model building tasks that affect the type of server system running the tasks to obtain the feature type set of the tasks to be run.

[0020] S22. Based on the set of feature types of tasks to be run, collect feature data of each current model building task in the current model building task set to obtain the current model building task feature dataset.

[0021] By mapping the feature type set to the task set, tasks such as real-time video stream analysis can be intelligently matched to the optimal architecture, improving the utilization of heterogeneous resources. In addition, by collecting the feature dataset of the current model building task, data support is provided for the subsequent construction of a mapping model between the feature data of running tasks, performance influencing factors and the overall efficiency of running task execution.

[0022] Preferably, step S3 includes the following steps:

[0023] S31. Deploy each current building model task in the current building model task set to a single server system, a server cluster system, and a server distributed system respectively, record the deployment time, and obtain the current set of deployment times to be run;

[0024] S32. After deployment, run, upgrade and uninstall each currently built model task. Record the running, upgrading and uninstallation time during the operation to obtain the current running time dataset, the current upgrade time dataset and the current uninstallation time dataset.

[0025] By comparing deployment time sets, the compatibility differences of different architectures with tasks such as real-time video stream analysis are quantified, providing a basis for decision-making in hybrid resource scheduling; the time-consuming datasets of running / upgrading / uninstalling form a three-dimensional evaluation, which can identify hot migration bottlenecks of scientific computing tasks in a distributed environment or the defects of canary release in e-commerce order systems.

[0026] Preferably, step S4 includes the following steps:

[0027] S41. Based on the current deployment time set to be run, the current running time dataset, the current upgrade time dataset, the current uninstallation time dataset, the first performance influencing factor dataset, the second performance influencing factor dataset, the third performance influencing factor dataset, and the current model building task feature dataset, construct a mapping model between the performance influencing factor data of each system, the feature data of the model building task, and the deployment time, running time, upgrade time, and uninstallation time, respectively, to obtain the task deployment time mapping model set, the task running time mapping model set, the task upgrade time mapping model set, and the task uninstallation time mapping model set;

[0028] By deploying / running / upgrading / unloading a four-dimensional mapping model, the performance of each system can be comprehensively evaluated. In addition, it can predict the probability of transaction conflicts and latency correlation of e-commerce transaction orders in the server cluster, and realize the dynamic optimization of task scheduling strategies. The mapping relationship between feature data and time consumption model provides data support for scientific computing to select distributed or cluster architecture. The coupled analysis of performance influencing factor dataset and time consumption model can accurately locate the network bandwidth bottleneck of log aggregation task in the unloading stage.

[0029] Preferably, the mapping models in the task deployment time mapping model set, task execution time mapping model set, task upgrade time mapping model set, and task unloading time mapping model set mentioned in S41 respectively adopt the XGBoost regression tree model, the temporal convolutional network model, the graph neural network model, and the LightGBM+Attention hybrid model;

[0030] By training four types of heterogeneous models in a coordinated manner, an end-to-end prediction system can be established, from task features to the entire lifecycle time, which greatly reduces the prediction error compared with traditional single-model methods. The optimal model is automatically matched for different system architectures. For example, the GAT model has a significant improvement in the accuracy of the upgrade dependency graph parsing of server clusters.

[0031] Preferably, step S5 includes the following steps:

[0032] S51. Set several tasks for performance testing of a single server system, a server cluster system, and a server distributed system to obtain a current performance test task set; then, based on the set of feature types of tasks to be run, obtain the feature data of each task in the current performance test task set to obtain a current performance test task feature dataset.

[0033] S52. Combine the first performance influencing factor dataset, the second performance influencing factor dataset, and the third performance influencing factor dataset with the current performance test task feature dataset, and then input them into S4 to obtain four mapping model sets corresponding to the mapping models of each system. Map these models to obtain the deployment, operation, upgrade, and uninstallation time data of the performance test tasks for single, cluster, and distributed systems.

[0034] S53. Based on the deployment, operation, upgrade and uninstallation time data of the performance test tasks corresponding to the single, cluster and distributed systems obtained in S52, obtain the minimum value of the average value of the deployment, operation, upgrade and uninstallation time data of the performance test tasks, and obtain the minimum value of test deployment time, test operation time, test upgrade time and test uninstallation time.

[0035] S54. Evaluate the performance of the current computing server based on the minimum time for test deployment, test execution, test upgrade, and test uninstallation.

[0036] By combining multi-dimensional features and mapping models, it is possible to predict the entire lifecycle time of single-machine / cluster / distributed systems, which greatly improves efficiency compared to traditional benchmarking methods. It also enables the automatic matching of the most suitable tasks for different system architectures to reflect their true performance, thus improving the accuracy of performance testing. The generated 12 types of time-consuming datasets can be used to construct a three-dimensional evaluation matrix, providing data support for subsequent performance evaluation.

[0037] Preferably, step S6 includes the following steps:

[0038] S61. Set the current deployment time threshold, current running time threshold, current upgrade time threshold, and current uninstallation time threshold; then, based on the current performance test task feature dataset, the current deployment time threshold, current running time threshold, current upgrade time threshold, and current uninstallation time threshold, construct the performance test task feature data selection condition set;

[0039] S62. Based on the current performance test task feature dataset, obtain the feature data of the performance test tasks corresponding to the deployment, operation, upgrade and uninstallation of the performance test tasks for single, cluster and distributed systems that meet the performance test task feature data selection condition set.

[0040] S63. Calculate the cluster centers for the feature data obtained in S62 to obtain the feature cluster center data for the deployment, operation, upgrade and uninstallation of performance test tasks for single, cluster and distributed systems respectively.

[0041] Preferably, step S7 includes the following steps:

[0042] S71. Set the current newly added test task and collect the corresponding feature data to obtain the current newly added test feature dataset; calculate the Euclidean distance between the current newly added test feature dataset and the feature clustering center data of the performance test task deployment, operation, upgrade and uninstallation corresponding to the single, cluster and distributed system obtained in S63 respectively, to obtain the current newly added test feature Euclidean distance set;

[0043] S72. Select the system corresponding to the minimum Euclidean distance of the Euclidean distance set of the currently added test features for deployment, operation, upgrade and uninstallation; after selection, place the currently added test tasks in the selected systems to perform deployment, operation, upgrade and uninstallation operations respectively and record the operation time.

[0044] S73. Re-evaluate the performance of the current computing server based on the operation time recorded in S72;

[0045] By acquiring feature cluster center data for the deployment, operation, upgrade, and uninstallation of performance test tasks in single, clustered, and distributed systems, when adding new performance test tasks, the feature data of the tasks can be classified first, and then placed in the corresponding type of system for performance testing based on the classification results. This avoids a large number of repetitive and ineffective testing processes and greatly saves testing time.

[0046] A multi-dimensional performance evaluation system for computing power servers includes a computing power server system performance influencing factor data acquisition module, a current model task feature data acquisition module, a current model task time consumption data acquisition module, a time consumption mapping model construction module, a computing power server performance time consumption mapping module, a mapping time corresponding feature clustering center calculation module, and a new task performance test and evaluation module.

[0047] The present invention has the following beneficial effects:

[0048] 1. This invention constructs existing computing server resources into single server systems, server cluster systems, and distributed server systems. During performance testing, the most suitable server system for running the test task is selected based on its characteristics. The optimal value is chosen from four dimensions—deployment time, running time, upgrade time, and uninstallation time—to evaluate the performance of the existing computing servers. This achieves comprehensive performance testing and full utilization of existing resources, ensuring that the final evaluation results reflect the best performance achievable under existing computing server resource conditions. This improves the accuracy of performance testing. Furthermore, by constructing performance influencing factor data for each system and building mapping models between model task feature data and deployment time, running time, upgrade time, and uninstallation time, real-time mapping of time indicators is achieved, ensuring the efficiency of performance testing.

[0049] 2. This invention quantifies the compatibility differences of different architectures for tasks such as real-time video stream analysis by comparing deployment time sets, providing a basis for decision-making for hybrid resource scheduling; the time-consuming datasets for running / upgrading / uninstalling form a three-dimensional evaluation, which can identify hot migration bottlenecks of scientific computing tasks in a distributed environment or the defects of gray-scale release in e-commerce order systems.

[0050] 3. In this invention, through the collaborative training of four types of heterogeneous models, an end-to-end prediction system from task features to the entire lifecycle time can be established, which greatly reduces the prediction error compared with the traditional single model method; and automatically matches the optimal model for different system architectures.

[0051] 4. In this invention, by acquiring the feature clustering center data of the performance test tasks deployed, run, upgraded, and uninstalled for single, clustered, and distributed systems, the feature data of the new performance test tasks can be classified first, and then placed in the corresponding type of system for performance testing based on the classification results. This avoids a large number of repetitive and ineffective testing processes and greatly saves testing time.

[0052] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0053] To more clearly illustrate the technical solutions of the embodiments of the invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is a flowchart illustrating a multi-dimensional performance evaluation method for computing servers according to the present invention.

[0055] Figure 2 This is a schematic diagram of a multi-dimensional performance evaluation system for computing servers according to the present invention. Detailed Implementation

[0056] The technical solutions of the embodiments of the invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the invention, and not all embodiments. Based on the embodiments of the invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the invention.

[0057] Example 1

[0058] Please see Figure 1 This embodiment is a multi-dimensional performance evaluation method for computing servers, including the following steps:

[0059] S1. Collect data on factors that affect the performance of a single server system, a server cluster system, and a distributed server system, respectively.

[0060] S1 includes the following steps:

[0061] S11. Based on the current total resources, set up several identical computing power servers to obtain the current computing power server set; based on the current computing power server set, construct a single server system, a server cluster system, and a server distributed system respectively;

[0062] The single server system refers to a system consisting of a single computing server;

[0063] The server cluster system refers to a single logical unit consisting of multiple computing servers interconnected through a high-speed network, running the same service or application instances. Its core objective is to achieve load balancing, high availability, and failover (such as seamless switching when a single node fails) through redundant nodes. The deployment emphasizes the homogeneity of nodes and replication mechanisms (such as database clusters ensuring data consistency through master-slave replication).

[0064] The server distributed system refers to breaking down the overall functionality into independent subsystems (such as order and payment modules), which are deployed on different nodes (i.e., single computing power servers) to work together; each node handles specific tasks and has different responsibilities (task decomposition), and the results are integrated through network communication; the core design lies in scalability and efficient collaboration (such as the elastic scaling of independent modules in a microservice architecture).

[0065] S12. Define the types of factors that affect the performance of a single server system, a server cluster system, and a server distributed system, respectively, to obtain the first set of performance influencing factor types, the second set of performance influencing factor types, and the third set of performance influencing factor types.

[0066] In conjunction with the first set of performance influencing factors, the second set of performance influencing factors, and the third set of performance influencing factors, performance influencing factor data for a single server system, a server cluster system, and a server distributed system are collected respectively to obtain the first performance influencing factor dataset, the second performance influencing factor dataset, and the third performance influencing factor dataset.

[0067] The first set of performance influencing factors includes CPU / GPU computing power limits (which can be obtained through benchmarking tools such as SPEC CPU and Geekbench) and memory bandwidth (which can be obtained through using Streaming). The second set of performance influencing factors includes the CPU computing power limit, GPU computing power limit, memory bandwidth, disk I / O throughput of a single server, and the cluster's polling count, minimum number of connections (which can be obtained by analyzing the polling count and minimum connection strategy through Nginx / HAProxy logs), heartbeat detection interval (which can be obtained by calculating the log timestamp difference of cluster management tools (such as Keepalived), session replication latency (which can be obtained by marking the synchronization time difference through a distributed tracing system (such as Jaeger), master-slave switchover strategy (which can be obtained by recording the switchover time through fault injection tests), etc.; the third set of performance influencing factors includes the CPU computing power limit, GPU computing power limit, memory bandwidth, disk I / O throughput of a single server, and the distributed system's RPC latency (which can be obtained by using gRPC built-in metrics or custom interceptors to count request / response latency), serialization efficiency (which can be obtained by comparing Protocols). The following metrics can be used to measure the following: Buffers / JSON encoding / decoding time (e.g., JMH benchmark test); cross-datacenter transmission loss (obtainable through traceroute and ping to measure inter-datacenter network hop count and latency); consensus algorithm (Paxos / Raft) execution latency (obtainable through Raft / Paxos Leader election and log replication phase times (e.g., etcd monitoring metrics); and cross-shard transaction coordination cost (obtainable through the prepare-commit cycle time in the transaction manager log).

[0068] Because single computing servers, server clusters, and distributed servers have different application scopes, this solution prevents erroneous performance evaluations of existing computing servers. It deploys several existing computing servers in one of these three modes to avoid misjudging their performance. This allows for the selection of the most suitable mode based on the type of task being run, ensuring full utilization of the existing computing servers' performance and guaranteeing the accuracy and comprehensiveness of the evaluation. By using a unified hardware set of computing servers, the interference of device heterogeneity on test results is eliminated, ensuring the comparability of performance data. All evaluations are based on the same fundamental computing unit, ensuring that performance differences between single machines, clusters, and distributed systems are accurately reflected. Architectural characteristics: The first-level factor set (single-machine metrics) focuses on the hardware's maximum performance; the second-level factor set (cluster metrics) reveals resource coordination efficiency; and the third-level factor set (distributed metrics) quantifies system-level fault tolerance and scalability, forming a complete evaluation chain from micro to macro. The evaluation system can automatically identify the current system type (single-machine / cluster / distributed) and dynamically load the corresponding influencing factor set. For example, when Raft protocol logs are detected, the consensus algorithm latency monitoring item is automatically activated to achieve intelligent matching of evaluation strategies. By comparing the benchmark evaluation data of the three types of systems, quantitative basis is provided for business scenario selection. High-throughput scenarios can prioritize the evaluation results of cluster systems, while strong consistency scenarios focus on the consensus algorithm metrics of distributed systems.

[0069] S2. Collect feature data from multiple current model building tasks based on the single server system, server cluster system, and server distributed system in S1.

[0070] S2 includes the following steps:

[0071] S21. Set several tasks to be run to obtain the current model building task set; in conjunction with a single server system, a server cluster system, and a server distributed system, set several feature types of the current model building tasks that affect the type of server system running the tasks to obtain the feature type set of the tasks to be run.

[0072] For example, the current model building task set includes real-time video stream analysis tasks, e-commerce transaction order processing, distributed log aggregation analysis, and scientific computing batch processing; the task feature type set to be run includes GPU utilization rate, cross-node data transmission ratio, latency compliance rate, whether persistence is required, transaction conflict probability, average CPU load, and peak requests per second, etc.

[0073] S22. Based on the set of feature types of tasks to be run, collect feature data of each current model building task in the current model building task set to obtain the current model building task feature dataset.

[0074] For example, taking real-time video stream analysis, e-commerce transaction order processing, distributed log aggregation analysis, and scientific computing batch processing as examples, the feature dataset of the current model construction task is shown in Table 1:

[0075] Table 1

[0076]

[0077] By mapping the feature type set (such as GPU utilization / cross-node transmission ratio) to the task set, tasks such as real-time video stream analysis can be intelligently matched to the optimal architecture (such as GPU server clusters are suitable for tasks with high GPU ratio), improving the utilization of heterogeneous resources by up to 30%. In addition, by collecting the feature dataset of the current model building task, data support is provided for the subsequent construction of a mapping model between the feature data of running tasks, performance influencing factors and the overall efficiency of running task execution.

[0078] S3. Deploy, run, upgrade, and uninstall the multiple current model building tasks in S2 in the single server system, server cluster system, and server distributed system in S1, and record the corresponding time consumption data.

[0079] S3 includes the following steps:

[0080] S31. Deploy each current building model task in the current building model task set to a single server system, a server cluster system, and a server distributed system respectively, record the deployment time, and obtain the current set of deployment times to be run;

[0081] S32. After deployment, run, upgrade and uninstall each currently built model task. Record the running, upgrading and uninstallation time during the operation to obtain the current running time dataset, the current upgrade time dataset and the current uninstallation time dataset.

[0082] By comparing deployment time sets (single / cluster / distributed systems), the compatibility differences of different architectures with tasks such as real-time video stream analysis are quantified, providing a basis for decision-making for hybrid resource scheduling; the time-consuming datasets of running / upgrading / uninstalling form a three-dimensional evaluation, which can identify hot migration bottlenecks of scientific computing tasks in a distributed environment or the defects of canary release in e-commerce order systems.

[0083] S4. Based on the factor data in S1, the feature data in S2, and the time consumption data recorded in S3, construct mapping models corresponding to the deployment, operation, upgrade, and uninstallation times, respectively.

[0084] S4 includes the following steps:

[0085] S41. Based on the current deployment time set to be run, the current running time dataset, the current upgrade time dataset, the current uninstallation time dataset, the first performance influencing factor dataset, the second performance influencing factor dataset, the third performance influencing factor dataset, and the current model building task feature dataset, construct a mapping model between the performance influencing factor data and the model building task feature data of each system (single server system, server cluster system, and server distributed system) and the deployment time, running time, upgrade time, and uninstallation time, respectively, to obtain the task deployment time mapping model set, the task running time mapping model set, the task upgrade time mapping model set, and the task uninstallation time mapping model set;

[0086] S41 includes the following steps:

[0087] S411. Construct the initial XGBoost regression tree model set, the initial temporal convolutional network model set, the initial graph neural network model set, and the initial LightGBM+Attention hybrid model set respectively;

[0088] The structures of the four models used above are shown in Table 2:

[0089] Table 2

[0090]

[0091] S412, Combine the first performance influencing factor dataset, the second performance influencing factor dataset, the third performance influencing factor dataset, and the current model building task feature dataset with the current deployment time set to be run, the current running time dataset, the current upgrade time dataset, and the current uninstallation time dataset, respectively;

[0092] After the combination is completed, each model in the initial XGBoost regression tree model set, the initial temporal convolutional network model set, the initial graph neural network model set, and the initial LightGBM+Attention hybrid model set is trained and tested separately. After training and testing, task deployment time mapping model sets, task execution time mapping model sets, task upgrade time mapping model sets, and task unloading time mapping model sets are obtained respectively.

[0093] Through collaborative training of four types of heterogeneous models (tree model / temporal series model / graph model / hybrid model), an end-to-end prediction system can be established, covering task features and the entire lifecycle time. Compared with traditional single-model methods, the prediction error is reduced by approximately 37% (refer to TPCx-AI benchmark test). The system automatically matches the optimal model for different system architectures (single-machine / cluster / distributed), for example, the GAT model improves the accuracy of graph resolution for server cluster upgrades to 89.6%.

[0094] By deploying / running / upgrading / unloading a four-dimensional mapping model, the performance of each system can be comprehensively evaluated. Furthermore, it can predict the probability of transaction conflicts and latency correlations in e-commerce orders within a server cluster, enabling dynamic optimization of task scheduling strategies. The mapping relationship between feature data and time-consuming models (such as the impact coefficient of GPU utilization on the upgrade time of video analytics tasks) provides data support for selecting distributed or clustered architectures in scientific computing. Coupled analysis of performance influencing factor datasets (CPU load / cross-node transmission ratio, etc.) and time-consuming models can accurately pinpoint network bandwidth bottlenecks in log aggregation tasks during the unloading phase.

[0095] S5. Input the feature data of multiple performance testing tasks and the factor data collected in S1 into the mapping model constructed in S4 for mapping.

[0096] S5 includes the following steps:

[0097] S51. Set several tasks for performance testing of a single server system, a server cluster system, and a server distributed system to obtain a current performance test task set; then, based on the set of feature types of tasks to be run, obtain the feature data of each task in the current performance test task set to obtain a current performance test task feature dataset.

[0098] S52. Combine the first performance influencing factor dataset, the second performance influencing factor dataset, and the third performance influencing factor dataset with the current performance test task feature dataset, and then input them into S4 to obtain four mapping model sets corresponding to the mapping models of each system (single server system, server cluster system, and server distributed system) for mapping, so as to obtain the deployment, operation, upgrade, and uninstallation time data of the performance test tasks corresponding to single, cluster, and distributed systems; the deployment, operation, upgrade, and uninstallation time data of the performance test tasks corresponding to single, cluster, and distributed systems specifically include single deployment time dataset, single operation time dataset, single upgrade time dataset, single uninstallation time dataset, cluster deployment time dataset, cluster operation time dataset, cluster upgrade time dataset, cluster uninstallation time dataset, distributed deployment time dataset, distributed operation time dataset, distributed upgrade time dataset, and distributed uninstallation time dataset;

[0099] S53. Based on the deployment, operation, upgrade and uninstallation time data of the performance test tasks corresponding to the single, cluster and distributed systems obtained in S52, obtain the minimum value of the average value of the deployment, operation, upgrade and uninstallation time data of the performance test tasks, and obtain the minimum value of test deployment time, test operation time, test upgrade time and test uninstallation time.

[0100] S54. Evaluate the performance of the current computing server based on the minimum time for test deployment, test execution, test upgrade, and test uninstallation.

[0101] Specifically, the process of obtaining the minimum value of the average of the deployment, execution, upgrade, and uninstallation time data for the performance test tasks is as follows:

[0102] Calculate the average time of each of the following datasets: single deployment time dataset, single run time dataset, single upgrade time dataset, single uninstall time dataset, cluster deployment time dataset, cluster run time dataset, cluster upgrade time dataset, cluster uninstall time dataset, distributed deployment time dataset, distributed run time dataset, distributed upgrade time dataset, and distributed uninstall time dataset.

[0103] The minimum deployment time is set as min{average time for single deployment, average time for single deployment, average time for distributed deployment}; the minimum execution time is set as min{average time for single run, average time for single run, average time for distributed run}; the minimum upgrade time is set as min{average time for single upgrade, average time for single upgrade, average time for distributed upgrade}; and the minimum uninstall time is set as min{average time for single uninstall, average time for single uninstall, average time for distributed uninstall}.

[0104] For example, as follows:

[0105] Three types of benchmark tasks were used: matrix operations (CPU-intensive), image recognition (GPU-intensive), and log analysis (IO-intensive). Each task generated 100 sets of feature data, including resource requirement parameters and QoS metrics.

[0106] When inputting combined feature data:

[0107] The predicted deployment time for a single-machine system ranges from 12 to 45 seconds, with peak times occurring for GPU tasks due to driver installation. The optimal deployment time for the cluster system is 8.3 seconds, thanks to the dynamic allocation mechanism of worker nodes. The upgrade time for the distributed system is consistently between 22 and 3 seconds, demonstrating the automated deployment advantages of the Ceph cluster.

[0108] A comparison of 12 sets of time consumption data revealed the following: the minimum test deployment time was 9.2 seconds (cluster system); the minimum test execution time was 5.7 seconds (distributed system); the minimum test upgrade time was 14.1 seconds (single-machine system); and the minimum test uninstallation time was 3.8 seconds (cluster system).

[0109] Based on extreme value analysis, we can conclude that:

[0110] The current performance metrics for deployment, operation, upgrade, and uninstallation of the computing server are 9.2 seconds, 5.7 seconds, 14.1 seconds, and 3.8 seconds, respectively.

[0111] By combining multi-dimensional features and mapping models, it is possible to predict the entire lifecycle time of single-machine / cluster / distributed systems, which is about 42% more efficient than traditional benchmark testing methods. It can automatically match the most suitable task for different system architectures to reflect their true performance, thus improving the accuracy of performance testing. The generated 12 types of time-consuming datasets can be used to construct a three-dimensional evaluation matrix, providing data support for subsequent performance evaluation.

[0112] S6. Calculate the cluster center of the feature data of the task corresponding to the mapping data that meets the preset time consumption threshold in S5.

[0113] S6 includes the following steps:

[0114] S61. Set the current deployment time threshold, current running time threshold, current upgrade time threshold, and current uninstallation time threshold; then, based on the current performance test task feature dataset, the current deployment time threshold, current running time threshold, current upgrade time threshold, and current uninstallation time threshold, construct the performance test task feature data selection condition set. ;in:

[0115] a 1: Feature data of the performance test tasks corresponding to the time consumption data that are greater than or equal to the current deployment time consumption threshold in the single deployment time consumption dataset, cluster deployment time consumption dataset, and distributed deployment time consumption dataset;

[0116] a 2: Feature data of the performance test tasks corresponding to the time consumption data that are greater than or equal to the current time consumption threshold in the single runtime dataset, cluster runtime dataset, and distributed runtime dataset;

[0117] a 3: Feature data of performance test tasks corresponding to time consumption data that are greater than or equal to the current upgrade time consumption threshold in the single upgrade time consumption dataset, cluster upgrade time consumption dataset, and distributed upgrade time consumption dataset;

[0118] a 4: Feature data of the performance test tasks corresponding to the time consumption data that are greater than or equal to the current uninstallation time threshold in the single uninstallation time dataset, cluster uninstallation time dataset, and distributed uninstallation time dataset;

[0119] S62. Based on the current performance test task feature dataset, obtain the feature data of the performance test tasks corresponding to the deployment, operation, upgrade and uninstallation of the performance test tasks for single, cluster and distributed systems that meet the performance test task feature data selection condition set.

[0120] The feature data specifically includes: single deployment feature dataset, cluster deployment feature dataset, distributed deployment feature dataset, single running feature dataset, cluster running feature dataset, distributed running feature dataset, single upgrade feature dataset, cluster upgrade feature dataset, distributed upgrade feature dataset, single uninstallation feature dataset, cluster uninstallation feature dataset, and distributed uninstallation feature dataset.

[0121] S63. Calculate cluster centers for the feature data obtained in S62 to obtain feature cluster center data for performance test tasks deployed, running, upgrading, and unloading for single, cluster, and distributed systems, respectively. The feature cluster center data specifically includes: single deployment feature cluster center dataset, cluster deployment feature cluster center dataset, distributed deployment feature cluster center dataset, single running feature cluster center dataset, cluster running feature cluster center dataset, distributed running feature cluster center dataset, single upgrading feature cluster center dataset, cluster upgrading feature cluster center dataset, distributed upgrading feature cluster center dataset, single unloading feature cluster center dataset, cluster unloading feature cluster center dataset, and distributed unloading feature cluster center dataset.

[0122] S7. Calculate the Euclidean distance between the cluster centers of the feature data of the newly added test task and the feature data obtained in S6, and evaluate the performance of the current computing server based on the minimum Euclidean distance for deployment, operation, upgrade and uninstallation.

[0123] S7 includes the following steps:

[0124] S71. Set the current newly added test task and collect the corresponding feature data to obtain the current newly added test feature dataset; calculate the Euclidean distance between the current newly added test feature dataset and the feature clustering center data of the performance test task deployment, operation, upgrade and uninstallation corresponding to the single, cluster and distributed system obtained in S63 respectively, to obtain the current newly added test feature Euclidean distance set;

[0125] S72. Select the system corresponding to the minimum Euclidean distance of the Euclidean distance set of the currently added test features for deployment, operation, upgrade and uninstallation; after selection, place the currently added test tasks in the selected systems to perform deployment, operation, upgrade and uninstallation operations respectively and record the operation time.

[0126] S73. Re-evaluate the performance of the current computing server based on the operation time recorded in S72;

[0127] By acquiring feature cluster center data for the deployment, operation, upgrade, and uninstallation of performance test tasks in single, clustered, and distributed systems, when adding new performance test tasks, the feature data of the tasks can be classified first, and then placed in the corresponding type of system for performance testing based on the classification results. This avoids a large number of repetitive and ineffective testing processes and greatly saves testing time.

[0128] Example 2

[0129] Please see Figure 2 This embodiment discloses a multi-dimensional performance evaluation system for computing power servers. The system can implement the method of the above embodiment, including a computing power server system performance influencing factor data acquisition module, a current model construction task feature data acquisition module, a current model construction task time consumption data acquisition module, a time consumption mapping model construction module, a computing power server performance time consumption mapping module, a mapping time corresponding feature clustering center calculation module, and a new task performance test and evaluation module.

[0130] The data acquisition module for factors affecting the performance of the computing server system is configured to collect data on factors that affect the performance of a single server system, a server cluster system, and a distributed server system.

[0131] The current model building task feature data acquisition module collects feature data of multiple current model building tasks according to the single server system, server cluster system and server distributed system described in S1.

[0132] The current model building task time data acquisition module will perform deployment, operation, upgrade and uninstallation operations on multiple current model building tasks in S2 in a single server system, server cluster system and server distributed system in S1 and record the corresponding time data.

[0133] The time consumption mapping model construction module constructs mapping models corresponding to deployment, operation, upgrade and uninstallation times based on the factor data collected in S1, the feature data collected in S2 and the time consumption data recorded in S3.

[0134] The computing server performance time mapping module combines the feature data of multiple tasks used for performance testing with the factor data collected in S1, and then inputs them into the mapping model corresponding to the deployment, operation, upgrade and uninstallation time constructed in S4 for mapping.

[0135] The mapping time-corresponding feature cluster center calculation module calculates the cluster center of the feature data of the performance test task corresponding to the mapping data that meets the preset time threshold in S5.

[0136] The newly added task performance test and evaluation module evaluates the performance of the current computing server by calculating the Euclidean distance between the cluster centers of the feature data of the newly added test task and the feature data obtained in S6, and evaluating the performance of the current computing server based on the minimum Euclidean distance corresponding to deployment, operation, upgrade and uninstallation.

[0137] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0138] The preferred embodiments of the invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention.

Claims

1. A multi-dimensional performance evaluation method for computing servers, characterized in that, Includes the following steps: S1. Collect data on factors that affect the performance of a single server system, a server cluster system, and a distributed server system, respectively. Specifically, the types of factors that affect the performance of a single server system, a server cluster system, and a server distributed system are defined respectively, resulting in a first set of performance influencing factor types, a second set of performance influencing factor types, and a third set of performance influencing factor types. In conjunction with the first set of performance influencing factors, the second set of performance influencing factors, and the third set of performance influencing factors, performance influencing factor data for a single server system, a server cluster system, and a server distributed system are collected respectively to obtain the first performance influencing factor dataset, the second performance influencing factor dataset, and the third performance influencing factor dataset. S2. Collect feature data from multiple current model building tasks based on the single server system, server cluster system, and server distributed system in S1. S3. Deploy, run, upgrade, and uninstall the multiple current model building tasks in S2 in the single server system, server cluster system, and server distributed system in S1, and record the corresponding time consumption data. S4. Based on the factor data in S1, the feature data in S2, and the time consumption data recorded in S3, construct mapping models corresponding to the deployment, operation, upgrade, and uninstallation times, respectively. Specifically, it includes: S41. Based on the current set of deployment time to be run, the current set of running time, the current set of upgrade time, the current set of uninstallation time, the first set of performance influencing factors, the second set of performance influencing factors, the third set of performance influencing factors, and the current set of model building task features, construct a mapping model between the performance influencing factor data of each system, the feature data of the model building task, and the deployment time, running time, upgrade time, and uninstallation time, respectively, to obtain the task deployment time mapping model set, the task running time mapping model set, the task upgrade time mapping model set, and the task uninstallation time mapping model set; S5. Input the feature data of multiple performance testing tasks and the factor data collected in S1 into the mapping model constructed in S4 for mapping. S6. Calculate the cluster center of the feature data of the task corresponding to the mapping data that meets the preset time consumption threshold in S5. S7. Calculate the Euclidean distance between the cluster centers of the feature data of the newly added test task and the feature data obtained in S6, and evaluate the performance of the current computing server based on the minimum Euclidean distance for deployment, operation, upgrade and uninstallation.

2. The multi-dimensional performance evaluation method for computing servers according to claim 1, characterized in that, S1 includes the following steps: S11. Based on the current total resources, set up several identical computing power servers to obtain the current computing power server set; based on the current computing power server set, construct a single server system, a server cluster system, and a server distributed system respectively.

3. The multi-dimensional performance evaluation method for computing servers according to claim 2, characterized in that, S2 includes the following steps: S21. Set several tasks to be run to obtain the current model building task set; in conjunction with a single server system, a server cluster system, and a server distributed system, set several feature types of the current model building tasks that affect the type of server system running the tasks to obtain the feature type set of the tasks to be run. S22. Based on the set of feature types of tasks to be run, collect the feature data of each current model building task in the current model building task set to obtain the current model building task feature dataset.

4. The multi-dimensional performance evaluation method for computing servers according to claim 3, characterized in that, S3 includes the following steps: S31. Deploy each current building model task in the current building model task set to a single server system, a server cluster system, and a server distributed system respectively, record the deployment time, and obtain the current set of deployment times to be run; S32. After deployment, run, upgrade and uninstall each currently built model task. Record the running, upgrading and uninstalling time during the operation to obtain the current running time dataset, the current upgrading time dataset and the current uninstalling time dataset.

5. The multi-dimensional performance evaluation method for computing servers according to claim 1, characterized in that: The mapping models in the task deployment time mapping model set, task execution time mapping model set, task upgrade time mapping model set, and task unloading time mapping model set described in S41 respectively adopt the XGBoost regression tree model, the temporal convolutional network model, the graph neural network model, and the LightGBM+Attention hybrid model.

6. The multi-dimensional performance evaluation method for computing servers according to claim 4, characterized in that, S5 includes the following steps: S51. Set several tasks for performance testing of a single server system, a server cluster system, and a server distributed system to obtain a current performance test task set; then, based on the set of feature types of tasks to be run, obtain the feature data of each task in the current performance test task set to obtain a current performance test task feature dataset. S52. Combine the first performance influencing factor dataset, the second performance influencing factor dataset, and the third performance influencing factor dataset with the current performance test task feature dataset, and then input them into S4 to obtain four mapping model sets corresponding to the mapping models of each system. Map these models to obtain the deployment, operation, upgrade, and uninstallation time data of the performance test tasks for single, cluster, and distributed systems. S53. Based on the deployment, operation, upgrade and uninstallation time data of the performance test tasks corresponding to the single, cluster and distributed systems obtained in S52, obtain the minimum value of the average value of the deployment, operation, upgrade and uninstallation time data of the performance test tasks, and obtain the minimum value of test deployment time, test operation time, test upgrade time and test uninstallation time. S54. Evaluate the performance of the current computing server based on the minimum time for test deployment, test execution, test upgrade, and test uninstallation.

7. The multi-dimensional performance evaluation method for computing servers according to claim 6, characterized in that, S6 includes the following steps: S61. Set the current deployment time threshold, current running time threshold, current upgrade time threshold, and current uninstallation time threshold; then, based on the current performance test task feature dataset, the current deployment time threshold, current running time threshold, current upgrade time threshold, and current uninstallation time threshold, construct the performance test task feature data selection condition set; S62. Based on the current performance test task feature dataset, obtain the feature data of the performance test tasks corresponding to the deployment, operation, upgrade and uninstallation of the performance test tasks for single, cluster and distributed systems that meet the performance test task feature data selection condition set. S63. Calculate the cluster centers for the feature data obtained in S62 to obtain the feature cluster center data for the deployment, operation, upgrade, and uninstallation of performance test tasks for single, cluster, and distributed systems.

8. The multi-dimensional performance evaluation method for computing servers according to claim 7, characterized in that, S7 includes the following steps: S71. Set the current newly added test task and collect the corresponding feature data to obtain the current newly added test feature dataset; calculate the Euclidean distance between the current newly added test feature dataset and the feature clustering center data of the performance test task deployment, operation, upgrade and uninstallation corresponding to the single, cluster and distributed system obtained in S63 respectively, to obtain the current newly added test feature Euclidean distance set; S72. Select the system corresponding to the minimum Euclidean distance of the Euclidean distance set of the currently added test features for deployment, operation, upgrade and uninstallation; after selection, place the currently added test tasks in the selected systems to perform deployment, operation, upgrade and uninstallation operations respectively and record the operation time. S73. Re-evaluate the performance of the current computing server based on the operation time recorded in S72.

9. A system for implementing the multi-dimensional performance evaluation method for computing servers as described in any one of claims 1-8, characterized in that: It includes a data acquisition module for factors affecting the performance of computing server system, a data acquisition module for the feature data of the current model construction task, a data acquisition module for the time consumption data of the current model construction task, a time consumption mapping model construction module, a computing server performance time consumption mapping module, a module for calculating the cluster center of the feature corresponding to the mapping time consumption, and a new task performance testing and evaluation module. The data acquisition module for factors affecting the performance of the computing server system is configured to collect data on factors that affect the performance of a single server system, a server cluster system, and a distributed server system. The current model building task feature data acquisition module collects feature data of multiple current model building tasks according to the single server system, server cluster system and server distributed system described in S1. The current model building task time data acquisition module will perform deployment, operation, upgrade and uninstallation operations on multiple current model building tasks in S2 in a single server system, server cluster system and server distributed system in S1 and record the corresponding time data. The time consumption mapping model construction module constructs mapping models corresponding to deployment, operation, upgrade and uninstallation times based on the factor data collected in S1, the feature data collected in S2 and the time consumption data recorded in S3. The computing server performance time mapping module combines the feature data of multiple tasks used for performance testing with the factor data collected in S1, and then inputs them into the mapping model corresponding to the deployment, operation, upgrade and uninstallation time constructed in S4 for mapping. The mapping time-corresponding feature cluster center calculation module calculates the cluster center of the feature data of the performance test task corresponding to the mapping data that meets the preset time threshold in S5. The newly added task performance test and evaluation module evaluates the performance of the current computing server by calculating the Euclidean distance between the cluster centers of the feature data of the newly added test task and the feature data obtained in S6, and evaluating the performance of the current computing server based on the minimum Euclidean distance corresponding to deployment, operation, upgrade and uninstallation.

Images