Evaluation device and method for coupling of application load calculation performance and storage performance

Abstract

The invention discloses a coupling evaluation device and method for application load calculation performance and storage performance. According to the technical scheme, the coupling evaluation method comprises the steps that a user interaction module is connected with a to-be-tested system environment and initializes the to-be-tested system environment; initializing a calculation performance and storage performance index system; the user interaction module receives a load starting command parameter, packages the parameter into a json file and sends the json file; the application load execution module executes an application load program and sends an index acquisition instruction; the data acquisition and storage module acquires and stores index data; and the evaluation module reads the acquired calculation performance and storage performance indexes from the time sequence database and calculates a coupling evaluation result of the calculation performance and the storage performance. According to the method, the real bottleneck of the overall performance of the system is revealed through coupling analysis of the calculation performance and the storage performance of the application load, so that a basis is provided for formulating a targeted optimization strategy.

Description

Technical Field

[0001] The present invention belongs to the technical field of electronic digital data processing, and further relates to an evaluation device and method for the coupling between computing performance and storage performance for application workloads in the technical field of computer hardware performance testing. The present invention can be used to analyze the coupling degree between the computing performance index and the storage performance index during the operation of the collected application workload. Through the coupling analysis, the true bottleneck of the overall system performance is revealed, and targeted optimization strategies are formulated. Background Art

[0002] In the fields of big data processing, cloud computing, and high-performance computing, computing performance and storage performance are two key performance indicators. Computing performance mainly determines the speed of data processing, while storage performance determines the read and write efficiency of data. The two are closely related. In the prior art, the evaluation of computing performance and storage performance is often carried out independently. Most traditional evaluation methods for computing and storage collect index data manually and analyze and calculate the index data to obtain the evaluation results of computing and storage respectively. This method requires manual participation in the entire evaluation process, resulting in low test efficiency.

[0003] In the patent document "An Automatic Detection Method and System for Server Hardware Performance Based on Benchmark Test Metrics" (application number 202210221840.5, publication number CN 114595132A) applied by Inspur Cloud Information Technology Co., Ltd., an automatic detection method and system for server hardware performance based on benchmark test metrics are proposed. The steps of the detection method include: first, reading the benchmark test metric library, remotely automatically obtaining the hardware configuration of the target server, generating an estimated value of the metric test, creating a server performance metric detection script, remotely executing the detection script, structurally processing the performance metric detection results, and combining them to generate a server performance detection report. The automatic detection system of the present invention realizes the automatic testing of server performance under the model of benchmark test metrics, obtains metric data and analyzes and processes the metric data, automatically generates a detection test script during the process, collects server information by combining with the BMC Web API interface of the application server, and automatically generates a test report, effectively meeting the requirements of the benchmark metric test of server hardware performance and realizing the rapid automation of the test process. However, the deficiencies of this technical solution still exist. The results remain at the level of single computing performance or storage performance, and it is impossible to identify whether the performance bottleneck is due to insufficient computing resources or storage performance limitations, making it difficult for the test results to reflect the performance in the real business scenario. In addition, this method only stays at static analysis, does not collect metric data during the execution of the application workload in real time, and cannot dynamically analyze the correlation between computing performance and storage performance.

[0004] Xidian University discloses an evaluation device and method for the throughput performance of HDFS in its patent document "An Evaluation Device and Method for the Throughput Performance of HDFS" (Application No. 202311430770.5, Publication No. CN 117421229 A). The evaluation device includes a cascaded user interaction module, a load execution module, a data collection and storage module, and an evaluation module, and the evaluation module is also connected to the user interaction module. The evaluation method includes: initializing the evaluation environment; establishing an index system for the throughput performance of HDFS, and the defined index system has 3 first-level indicators and 8 second-level indicators; the user interaction module encapsulates a json file and sends it; the load execution module executes a load program and sends an index collection instruction; the data collection and storage module collects second-level index data and stores it; the evaluation module obtains the comprehensive evaluation result of the throughput performance of HDFS. The deficiencies of this technical solution are that the defined index system fails to cover the key factors affecting the throughput performance of HDFS, fails to fully and comprehensively characterize the computing performance and storage performance, cannot comprehensively reflect the performance of computing and storage, resulting in inaccurate results, and fails to fully capture the details that may affect performance. Summary of the Invention

[0005] The object of the present invention is to propose an evaluation device and method for the coupling between the computing performance and storage performance of application loads in view of the defects existing in the above-mentioned prior art, aiming to solve the problems existing in the prior art that the test object only targets a single computing performance or storage performance level, the performance characterization index system is single, and the result cannot identify whether the performance bottleneck is due to insufficient computing resources or storage performance limitations, making the test result difficult to reflect the performance in the real business scenario and having insufficient characterization ability, as well as the problem of being unable to collect index data during the execution of computing tasks in real time.

[0006] The technical idea for achieving the object of the present invention is that the method of the present invention constructs an index system for characterizing computing performance and storage performance, including system-level computing performance and storage performance indexes covering multiple aspects of the system CPU, memory, and disk I / O, and process-level computing performance and storage performance indexes covering process CPU, memory, and disk I / O, so as to comprehensively and fully characterize computing performance and storage performance, and solve the problems of single index system and incomplete performance characterization existing in the prior art; the data acquisition and storage module is used to collect in real time the indexes of computing performance and storage performance during the execution of the application load program, and persistently store them in the time series database, so as to solve the problems of inability to collect index data during the execution of the application load in real time and inability to dynamically analyze the correlation between computing performance and storage performance; the evaluation module reads the collected computing performance indexes and storage performance indexes from the time series database, analyzes and processes the data, identifies the correlation between computing and storage indexes, and calculates the coupling degree between the overall computing performance and storage performance, so as to solve the problem that only a single computing performance or storage performance level is considered separately, and it is impossible to identify whether the performance bottleneck is due to insufficient computing resources or storage performance limitations.

[0007] To achieve the above object, the evaluation device of the present invention includes a user interaction module, an application load execution module, an evaluation module, a data acquisition and storage module, and a computing platform for coupling evaluation; wherein:

[0008] The user interaction module is used to receive the information of the system to be tested and the application load start command parameters input by the user. The user interaction module includes a unit for verifying the information of the system to be tested, a unit for verifying the application load start command parameters, and a connection and sending unit; wherein, the unit for verifying the information of the system to be tested is used to verify the format and connectivity of the IP address, port number, user name, and password in the information of the system to be tested to ensure the reliability of accessing the system to be tested. The unit for verifying the application load start command parameters verifies the format of the application load start command; after receiving the user input and passing the verification, the connection and sending unit connects the information of the system to be tested input by the user to the system to be tested by means of network communication, encapsulates the received application load start command parameters into the json file format, and sends the encapsulated start command parameters to the load execution module of the system to be tested by means of network communication;

[0009] The application load execution module is used to receive the application load start command parameters of the user interaction module and execute the application load according to the application load start command; after receiving the start command parameters of the user interaction module, it splits the application load start parameters in the json file according to a preset delimiter, and constructs a test script according to the split multiple parameters; then, it starts the application load program by executing the test script, and at the same time sends an index collection instruction to the data acquisition and storage module;

[0010] The data acquisition and storage module includes a data acquisition unit and a data storage unit, which are used to receive the metric acquisition instructions from the application load execution module, and collect and store the metric data. After receiving the metric acquisition instructions sent by the application load execution module, the data acquisition unit starts to collect the metrics of the computing performance and storage performance in real time during the execution of the application load program. The data storage unit persistently stores the data collected by the data acquisition unit into the time series database.

[0011] The evaluation module is used to perform a coupling evaluation on the collected computing performance metric data and storage performance metric data. When the application load execution is completed, the evaluation module receives the coupling evaluation instruction from the user interaction module, reads the collected computing performance metrics and storage performance metrics from the time series database, aligns the collected data according to the timestamp or event, ensures that the computing performance and storage performance metrics can be analyzed for correlation, performs statistical analysis on the aligned data, identifies the correlation between the computing performance and storage performance metrics, normalizes the data with different dimensions to make the subsequent analysis more scientific and accurate, calculates the coupling degree between the overall computing performance and storage performance, generates a report and sends it back to the user interaction module for result display.

[0012] The steps of an evaluation method for the coupling of application load computing performance and storage performance of the present invention are as follows:

[0013] Step 1, the user interaction module connects to and initializes the system environment to be tested.

[0014] Step 2, initialize the metric systems of system-level computing performance metrics, storage performance metrics, process-level computing performance metrics, and storage performance metrics.

[0015] Step 3, the user interaction module receives the load start command parameters and encapsulates them into a json file for sending.

[0016] Step 4, the application load execution module executes the application load program and sends metric acquisition instructions.

[0017] Step 5, the data acquisition and storage module collects and stores the metric data.

[0018] Step 6, the evaluation module calculates the coupling evaluation result of computing and storage.

[0019] Further, the system environment to be tested means that, through network communication, the user interaction module and the evaluation module are initialized and deployed on the coupling evaluation device and started; through network transmission, the application load is installed and deployed in the system to be tested, and through network communication, the data acquisition and storage module is installed and deployed on the system to be tested.

[0020] Further, the system-level computing performance metrics and storage performance metrics include system-level metrics of the system CPU, system memory, and system disk. The process-level computing performance metrics and storage performance metrics include process-level metrics of process CPU, process memory, and process disk I / O.

[0021] Further, the user interaction module receiving the load start command parameter and encapsulating it into a json file for sending means that the user interaction module receives the application load start command parameter input by the user, including an application load start command parameter verification unit and a connection sending unit; the application load start command parameter verification unit receives the application load start command parameter input by the user and performs verification, and the connection sending unit connects to the system under test using the system information to be measured input by the user, and through network communication, encapsulates the application load start command parameter into a json file and sends it to the application load execution module of the system under test.

[0022] Further, the steps for the application load execution module to execute the application load program and send an index collection instruction are as follows:

[0023] First step, the application load execution module splits the application start parameters in the json file according to a preset delimiter; the preset delimiter is set according to the JSON parameter format, using the pipe character "|" as the first-level delimiter and the semicolon ";" as the second-level delimiter.

[0024] Second step, form all the application load start parameters obtained by splitting into an executable Shell test script;

[0025] Third step, execute the application load program through the Shell test script, and send an index collection instruction to the data collection and storage module while executing the application load program.

[0026] Further, the collection of index data means that after the data collection and storage module receives the index collection instruction sent by the collection and storage module, it collects the metrics of computing performance and storage performance in real time during the execution of the load program, and persists and stores them in the time series database.

[0027] The steps for obtaining the coupling evaluation result of computing performance and storage performance are as follows:

[0028] First step, the evaluation module reads the collected system-level computing performance and storage performance metrics and process-level computing performance and storage performance metrics from the time series database, aligns the collected data by timestamp or event, and data alignment means aligning the monitoring data from different time periods and different sources at the same time point;

[0029] In the second step, perform statistical analysis on the aligned data, standardize the timestamps, and standardize the timestamp formats and data precisions of each data source.

[0030] In the third step, address the issue of inconsistent collection frequencies from different data sources. Through interpolation and resampling techniques, achieve data alignment at unified time points.

[0031] In the third step, by calculating the correlation coefficient matrix between each performance indicator in the collected data, identify pairs of indicators with collinearity and redundant indicator pairs with high correlation; for the identified redundant indicators, perform reasonable processing through principal component analysis (PCA) and feature selection.

[0032] In the fourth step, calculate the coupling degree between the overall computing performance and storage performance. Use the entropy method to calculate the index weights for computing performance indicators and storage performance indicators respectively. Use a comprehensive evaluation model to calculate the comprehensive evaluation index of computing performance indicators and storage performance indicators. Use a coupling degree model to calculate the coupling relationship between performance and storage performance.

[0033] In the fifth step, based on the results of the coupling analysis, generate a report and display it through the user interaction module.

[0034] The calculation of the index weights for computing performance indicators and storage performance indicators respectively using the entropy method is obtained through the following steps:

[0035] In the first step, calculate the degree of variability of each indicator:

[0036]

[0037] where \(P_{ij}\) represents the proportion of the value of the \(j\)-th indicator in the \(i\)-th group of samples, \(x_{ij}\) represents the value of the \(j\)-th indicator after data alignment and correlation analysis for the \(i\)-th group of samples, \(n\) represents the total number of sample groups in computing performance or storage performance, and \(m\) represents the total number of indicators included in each sample.

[0038] In the second step, calculate the information entropy value of each indicator:

[0039]

[0040] where \(E\) j represents the information entropy value of the \(j\)-th indicator, \(\ln\) represents the logarithmic operation with the natural constant \(e\) as the base, and \(\sum\cdot\) represents the summation operation.

[0041] In the third step, calculate the weight of each indicator:

[0042]

[0043] where \(W\) j represents the weight of the \(j\)-th indicator.

[0044] Furthermore, the comprehensive evaluation index is obtained by the following formula:

[0045]

[0046] wherein, represents the comprehensive evaluation index of the i-th group of samples in the k-th performance, k = 1, 2, u1 represents the comprehensive evaluation index of computing performance, and u2 represents the comprehensive evaluation index of storage performance.

[0047] Furthermore, the coupling relationship between the computing performance and the storage performance is obtained by the following formula:

[0048]

[0049] wherein, C represents the coupling degree between the computing performance and the storage performance, and its value ranges from [0, 1]. The closer the C value is to 1, the stronger the correlation between the computing performance and the storage performance and the more ordered it tends to be. On the contrary, the weaker the correlation between the computing performance and the storage performance and the more disordered it tends to be.

[0050] Compared with the prior art, the present invention has the following advantages:

[0051] First, the method of the present invention establishes an index system for characterizing computing performance and storage performance, including a hierarchical index system of system-level indicators (CPU utilization rate, memory bandwidth, disk I / O throughput) and process-level indicators (thread context switching frequency, process I / O waiting time), covering multiple levels such as hardware resources, operating systems, and application loads, overcoming the defects of a single detection index system and incomplete characterization of computing performance and storage performance in the prior art, enabling the present invention to more comprehensively and accurately characterize computing performance and storage performance and improving the accuracy of detection.

[0052] Second, the method of the present invention quantifies the coupling degree between computing performance and storage performance during the execution of application loads, reveals the interaction mechanism between computing and storage, uses normalization processing to eliminate dimension differences, and calculates the dynamic coupling degree of computing and storage indicators through statistical analysis methods, overcoming the problem in the prior art that only a single computing performance or storage performance level is compared, and it is impossible to identify whether the performance bottleneck is due to insufficient computing resources or storage performance limitations, enabling the present invention to reveal the true bottleneck of the overall system performance through the analysis of the coupling between computing performance and storage performance of application loads and formulate corresponding optimization strategies targeted.

[0053] Third, the data acquisition module in the detection device of the present invention can collect the computing performance and storage performance indicators during the execution of the application load in real time, and align them based on timestamps or event triggers to ensure the synchronization of data in the time dimension. It overcomes the deficiency of static testing in the prior art that cannot dynamically analyze the correlation between the CPU during high-load periods and the storage I / O peak, enabling the present invention to collect the indicator data during the execution of the computing task in real time while the application load is running, providing a reference basis for real-time dynamic analysis of the computing performance and storage performance of the application load. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 is a schematic structural diagram of the evaluation device of the present invention;

[0055] Figure 2 is a flowchart of the evaluation method of the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0056] The present invention will be further described in detail below with reference to the accompanying drawings.

[0057] Referring to Figure 1 , the evaluation device of the present invention will be further described.

[0058] The evaluation device of the present invention includes a cascaded user interaction module, an application load execution module, a data acquisition and storage module, and an evaluation module; the input end of the evaluation module is also connected to the output end of the user interaction module for realizing the coupling evaluation of computing and storage capabilities.

[0059] The user interaction module includes a test system information verification unit, an application load start command parameter verification unit, and a connection and sending unit. This module is used to receive the test system information and application load start command parameters input by the user. Among them, the test system information verification unit is used to verify the format and connectivity of the IP address, port number, username, and password in the test system information to ensure the reliability of accessing the test system. The application load start command parameter verification unit verifies the format of the application load start command. After receiving the user input and passing the verification, the connection and sending unit connects to the test system according to the test system information input by the user using the network communication method, encapsulates the received application load start command parameters into the json file format, and sends the encapsulated start command parameters to the load execution module of the test system using the network communication.

[0060] The application load execution module is used to receive the application load start command parameters from the user interaction module and execute the application load according to the application load start command. After receiving the start command parameters from the user interaction module, it splits the application load start parameters in the json file according to a preset delimiter, constructs a test script based on the split multiple parameters, starts the application load program by executing the test script, and simultaneously sends a metric collection instruction to the data collection and storage module.

[0061] The data collection and storage module includes a data collection unit and a data storage unit. This module is used to receive the metric collection instruction from the application load execution module and collect and store the metric data. Among them, after receiving the metric collection instruction sent by the application load execution module, the data collection unit starts to collect the metrics of the computing performance and storage performance in real time when the application load program is executed. The data storage unit persistently stores the data collected by the data collection unit into the time series database.

[0062] The evaluation module is used to perform a coupling evaluation on the collected computing performance metric data and storage performance metric data. After the application load execution is completed, the evaluation module receives the coupling evaluation instruction from the user interaction module, reads the collected computing performance metrics and storage performance metrics from the time series database, aligns the collected data by timestamp or event to ensure that the computing performance and storage performance metrics can be analyzed for correlation. Statistically analyze the aligned data to identify the correlation between the computing performance and storage performance metrics. Normalize the data with different dimensions to make the subsequent analysis more scientific and accurate. Calculate the coupling degree between the overall computing performance and storage performance, generate a report and send it back to the user interaction module for result display.

[0063] Refer to Figure 2 , and further describe the steps of the evaluation method of the present invention.

[0064] Step 1, the user interaction module connects to and initializes the system environment to be tested.

[0065] The user interaction module of the coupling evaluation device receives the information of the system to be tested input by the user, connects to the system to be tested through network communication, and initializes the coupling evaluation device and the system environment to be tested, including initializing and deploying the user interaction module and the evaluation module on the coupling evaluation device and starting them, installing and deploying the application load in the system to be tested through network transmission, and installing and deploying the data collection and storage module on the system to be tested through network communication

[0066] Step 2, initialize the computing performance and storage performance metric systems.

[0067] Initialize the data acquisition and storage module installed in the system to be tested. The initialization includes an index system covering system-level computing performance and storage performance metrics in multiple aspects such as the system CPU, memory, and disk I / O, and process-level computing performance and storage performance metrics covering process CPU, memory, and disk I / O.

[0068] In the embodiments of the present invention, the numbers of system-level metrics and process-level metrics are 35 and 21 respectively, where:

[0069] The 35 system-level computing performance and storage performance metrics include system CPU metrics, system memory metrics, and system disk metrics as follows:

[0070] The system CPU metrics collected in the embodiments of the present invention include 11: total CPU usage rate, CPU usage rate of each core, user-mode CPU usage rate, system-mode CPU usage rate, CPU idle rate, proportion of CPU hard interrupt time, proportion of CPU soft interrupt time, number of CPU context switches, maximum value of CPU frequency, minimum value of CPU frequency, CPU scaling frequency.

[0071] The system memory metrics collected in the embodiments of the present invention include 10: memory usage rate, available memory of the node, amount of cached memory used, swap memory usage rate, number of swap memory bytes, amount of memory buffer used, active memory, inactive memory, memory page cache, dirty memory.

[0072] The system disk metrics collected in the embodiments of the present invention include 14: disk read rate, disk write rate, disk usage rate, disk read latency, disk write latency, disk I / O operation time, disk queue length, disk usage percentage, number of disk read bytes, number of disk written bytes, disk read time, disk write time, disk wait time, disk usage capacity.

[0073] The 21 process-level computing performance and storage performance metrics include: process CPU metrics, process memory metrics, and process disk metrics as follows:

[0074] The process CPU metrics collected in the embodiments of the present invention include 7: process CPU usage rate, process user-mode CPU usage rate, process system-mode CPU usage rate, process CPU time, number of process context switches, number of process voluntary context switches, number of process involuntary context switches.

[0075] The process-level memory metrics collected in the embodiments of the present invention include 7: amount of process memory used, amount of process virtual memory used, amount of process anonymous memory used, amount of process shared memory used, number of process memory page faults, number of process primary memory page faults, number of process secondary memory page faults.

[0076] The process-level disk metrics collected in the embodiments of the present invention include four: the number of bytes read from the disk by the process, the number of bytes written to the disk by the process, the disk read operation rate of the process, and the disk write operation rate of the process.

[0077] Step 3, the user interaction module receives the load startup command parameters and encapsulates them into a JSON file for sending:

[0078] The user interaction module will receive the application load startup command parameters input by the user, including an application load startup command parameter verification unit and a connection sending unit.

[0079] The application load startup command parameter verification unit receives the application load startup command parameters input by the user and performs verification. The semantic analysis engine uses JSON Schema to define the verification rules and logical validators to check the validity of the parameter combination. For example: when workload_type = "compute", the "executor_cores" parameter must be included.

[0080] The connection sending unit connects to the system under test using the system information under test input by the user, and encapsulates the application load startup command parameters into a JSON file and sends it to the application load execution module of the system under test through network communication.

[0081] Step 4, the application load execution module executes the application load program and sends an index collection instruction.

[0082] The application load execution module splits the application startup parameters in the JSON file according to a preset delimiter, constructs a test script based on the multiple load parameters obtained by the split, then executes the application program through the test script, and at the same time sends an index collection instruction to the data collection and storage module.

[0083] The specific method for encapsulating multiple application load parameters into a JSON file in the embodiments of the present invention is as follows: the user interaction module creates a data structure, fills the load parameters into the data structure to ensure that the fields and values in the data structure conform to the syntax rules of JSON. Then, the data structure is converted into a JSON file through the JSON method library. The specific method for executing the application load program is as follows: the application load execution module creates a shell script for load execution according to the IP address of the system under test and the test data parameter specifications in the application load parameters, and then sends an execution command to the IP address of the system under test through the shell script.

[0084] Step 5, the data collection and storage module collects and stores the index data.

[0085] The data acquisition and storage module collects the metrics of computing performance and storage performance during the execution of the load program in real time according to the received metric acquisition instructions, and persists and stores them in the time series database;

[0086] In the embodiments of the present invention, the metric collector node_exporter is used to collect system-level computing performance and storage performance metric data, the process_exporter is used to collect process-level computing performance and storage performance metric data, and the time series database Prometheus is used to store system-level and process-level computing performance and storage performance metric data. Node_exporter and process_exporter need to be installed on the system to be tested, and Prometheus needs to be installed on the coupling evaluation device.

[0087] The data acquisition and storage module consists of a collector, an aggregator, a time series database, a monitoring backend, and a monitoring frontend. The functions of each component are as follows:

[0088] The collector consists of node_exporter and process_exporter, and is used to collect various computing performance and storage performance metrics at the system level and process level. Each collector is an independent program application responsible for collecting the required metric data from the monitoring target (such as a server or an application).

[0089] The aggregator is the core component for collecting data from all collectors. It is responsible for pulling data from multiple collectors, and aggregating and processing these data.

[0090] The time series database is used to save the monitoring metric data aggregated by the aggregator. The time series database is specifically designed to process time series data and can efficiently store and query large-scale monitoring data.

[0091] The monitoring backend provides RESTful interfaces for external entities to access the data of the aggregator conveniently and securely. The monitoring backend is responsible for processing external data requests, querying the aggregator, and returning the corresponding data results.

[0092] The monitoring frontend is an interface for displaying and visualizing monitoring data. Users can view real-time system and process-level monitoring metrics through the monitoring frontend and perform corresponding analysis and operations.

[0093] Step 6, the evaluation module calculates the coupling evaluation results of computing and storage.

[0094] The evaluation module reads the collected system-level metrics and process-level computing performance and storage performance metrics from the time series database, aligns the collected data by timestamp or event, and ensures that the computing performance and storage performance metrics can be analyzed for correlation. Statistical analysis is performed on the aligned data to identify the correlation between the computing performance and storage performance metrics. The data with different dimensions is normalized to make subsequent analysis more scientific and accurate. Calculate the coupling degree between overall computing and storage, generate a report, and display it through the user interaction module.

[0095] Step 6.1, data alignment refers to aligning monitoring data from different time periods and different sources at a unified time point for accurate analysis and comparison, which is crucial for performance monitoring, bottleneck identification, and resource optimization.

[0096] Step 6.1.1, perform timestamp standardization to standardize the timestamp formats and data precisions of each data source to ensure that the data can be compared on the same time basis.

[0097] Assume the original timestamp is t raw , and the standardized timestamp is t std , then:

[0098] T std = convert toUTC(traw)

[0099] T std = round(t std , precision)

[0100] where convert_to_UTC is the time conversion function and precision is the time precision (such as milliseconds).

[0101] Step 6.1.2, handle the problem of inconsistent collection frequencies of different data sources, and achieve data alignment at a unified time point through interpolation and resampling techniques.

[0102] Assume the original timestamp is t std , and the adjusted timestamp is t adj , then:

[0103] t adj = round(t std , precision)

[0104] where precision is the unified time precision (such as milliseconds).

[0105] For high-frequency data, use the resampling method to downsample it to the target frequency.

[0106] Assume that the high-frequency data acquisition frequency is f high , and the target frequency is f target , then the data value after resampling

[0107] x resample is:

[0108]

[0109] where i.e., the number of data points within each time window.

[0110] For low-frequency data, an interpolation algorithm is used to interpolate it to the target frequency.

[0111] Assume that the low-frequency data points are (t low , x low ), and the target-frequency data points are (t target , x target ). Using the linear interpolation method, then:

[0112]

[0113] where t i ≤ t < t i+1 , and t is the target time point.

[0114] Through resampling and interpolation algorithms, the problems of large data volume and complex processing can be effectively solved. The resampling method reduces the acquisition frequency of high-frequency data and the number of data points, thereby reducing the computational complexity of data processing. The interpolation algorithm aligns all data sources at a unified frequency by raising low-frequency data to the target frequency, facilitating subsequent analysis and processing.

[0115] Step 6.2: The collected data often contains a large number of performance indicators, and there may be significant linear correlations among these indicators. Directly importing these linearly correlated data into subsequent algorithm modules will not only consume a large amount of computing resources but may also cause redundant information to affect the accuracy of the analysis results. Therefore, performing collinearity analysis on the collected data is an essential step, which can significantly reduce the pressure on subsequent algorithm operations.

[0116] Calculate the correlation coefficient matrix: Calculate the correlation coefficient matrix among the performance indicators in the collected data to identify pairs of indicators with collinearity.

[0117] For two performance indicators X and Y, the calculation formula for their Pearson correlation coefficient r is:

[0118]

[0119] where and They are the means of X and Y respectively.

[0120] Identifying redundant indicators: Based on the calculated correlation coefficient matrix, identify pairs of indicators with high correlation, and consider one of the indicators in these pairs to be redundant.

[0121] Method: Set a correlation coefficient threshold (such as 0.8), and indicator pairs above this threshold are considered to have collinearity.

[0122] Steps: Traverse the correlation coefficient matrix to identify pairs of indicators whose absolute value of the correlation coefficient is greater than the threshold.

[0123] Processing redundant indicators: For the identified redundant indicators, they cannot be simply deleted, but need to be reasonably processed to retain the main information of the data. Common methods include principal component analysis (PCA) and feature selection.

[0124] The so-called principal component analysis means that assuming the original data matrix is X, PCA obtains the eigenvector matrix W through eigenvalue decomposition, and projects X into a low-dimensional space:

[0125] Z = XW

[0126] Among them, W is the eigenvector matrix, and Z is the data after dimensionality reduction.

[0127] The so-called feature selection means that the information gain IG is used to measure the importance of a certain feature A for the result variable Y, and its calculation formula is:

[0128] IG(Y, A) = H(Y) - H(Y|A)

[0129] Among them, H(Y) is the entropy of the result variable Y, and H(Y|A) is the conditional entropy of Y under the condition that the feature A is given.

[0130] Through the above specific solutions, redundant performance indicators in monitoring data can be effectively identified and processed. By using the correlation coefficient matrix calculation, principal component analysis (PCA) and feature selection methods, not only can redundant information be reduced, but also the main data information can be retained, significantly improving the accuracy and calculation efficiency of data analysis.

[0131] Step 6.3, Coupling calculation.

[0132] Step 6.3.1, Calculate the indicator weights.

[0133] Using the entropy method based on real data, the weight allocation is more scientific and objective, with higher requirements for the accuracy and integrity of the data, and the calculation process is relatively complex.

[0134] Assume that the system has a total of n samples, and each sample contains m indicators. Calculate the indicator weights for the calculation and storage systems respectively.

[0135] Step 1: Calculate the variability degree of each indicator:

[0136]

[0137] Among them, P ij represents the proportion of the value of the j-th indicator in the i-th group of samples, and x ij represents the value of the j-th indicator after data alignment and correlation analysis of the i-th group of samples. n represents the total number of sample groups in computing performance or storage performance, and m represents the total number of indicators included in each sample;

[0138] Step 2: Calculate the information entropy value of each indicator:

[0139]

[0140] Among them, E j represents the information entropy value of the j-th indicator, ln represents the logarithmic operation with the natural constant e as the base, and ∑· represents the summation operation;

[0141] Step 3: Calculate the weight of each indicator:

[0142]

[0143] Among them, W j is the weight of the j-th indicator.

[0144] Step 6.3.2: Calculate the comprehensive evaluation index.

[0145] Use the comprehensive evaluation model to calculate the comprehensive evaluation indexes of the computing system and the storage system, and illustrate the development levels of each subsystem through the comprehensive evaluation indexes of each system. The calculation formula is as follows:

[0146]

[0147] Among them, represents the comprehensive evaluation index of the i-th group of samples in the k-th performance, k = 1, 2, u1 represents the comprehensive evaluation index of computing performance, and u2 represents the comprehensive evaluation index of storage performance.

[0148] Step 6.3.3: Calculate the coupling degree.

[0149] The coupling degree mainly reflects the intensity of the interaction and mutual influence between computing performance and storage performance. Use the coupling degree model to calculate the coupling relationship between computing performance and storage performance. The binary coupling degree calculation formula is as follows:

[0150]

[0151] Among them, C represents the coupling degree of computing performance and storage performance, with a value range of [0, 1]. The closer the C value is to 1, the stronger the correlation between computing performance and storage performance and the more ordered it tends to be. Conversely, the weaker the correlation between computing performance and storage performance and the more disordered it tends to be.

Claims

1. An evaluation device for the coupling of application load computing performance and storage performance, characterized in that, The evaluation device includes a user interaction module, an application load execution module, an evaluation module, a data collection and storage module, and a computing platform for coupling evaluation; where: The user interaction module is used to receive the information of the system under test and the application load start command parameters input by the user. The user interaction module includes a system under test information verification unit, an application load start command parameter verification unit, and a connection and sending unit; among them, the system under test information verification unit is used to verify the format and connectivity of the IP address, port number, username, and password in the system under test information to ensure the reliability of accessing the system under test. The application load start command parameter verification unit verifies the format of the application load start command; after receiving the user input and passing the verification, the connection and sending unit uses the network communication method to connect the information of the system under test input by the user to the system under test, encapsulates the received application load start command parameters into the json file format, and sends the encapsulated start command parameters to the load execution module of the system under test through network communication; The application load execution module is used to receive the application load start command parameters of the user interaction module and execute the application load according to the application load start command; after receiving the start command parameters of the user interaction module, it splits the application load start parameters in the json file according to the preset delimiter, and constructs a test script according to the multiple split parameters; then, it starts the application load program by executing the test script, and at the same time sends an index collection instruction to the data collection and storage module; The data collection and storage module includes a data collection unit and a data storage unit, which are used to receive the index collection instruction of the application load execution module and collect and store the index data; after receiving the index collection instruction sent by the application load execution module, the data collection unit starts to collect the indexes of the computing performance and storage performance in real time when the application load program is executed; the data storage unit persistently stores the data collected by the data collection unit into the time series database; The evaluation module is used to perform coupling evaluation on the collected computing performance index data and storage performance index data; when the application load execution is completed, the evaluation module receives the coupling evaluation instruction of the user interaction module, reads the collected computing performance index and storage performance index from the time series database, aligns the collected data according to the time stamp or event, and ensures that the computing performance and storage performance indexes can be analyzed for correlation; performs statistical analysis on the aligned data to identify the correlation between the computing performance and storage performance indexes; normalizes the data with different dimensions to make the subsequent analysis more scientific and accurate; calculates the coupling degree between the overall computing performance and storage performance, generates a report and sends it back to the user interaction module for result display.

2. The evaluation method for coupling performance of application load calculation performance and storage performance of the evaluation device according to claim 1, characterized in that The steps of this evaluation method are as follows: Step 1, the user interaction module connects to and initializes the system environment under test; Step 2, initialize the index system of system-level computing performance indexes, storage performance indexes, process-level computing performance indexes, and storage performance indexes; Step 3: The user interaction module receives the load startup command parameters, encapsulates them into a JSON file, and sends it. Step 4: The application load execution module executes the application load program and sends an index collection instruction. Step 5: The data collection and storage module collects the index data and stores it. Step 6: The evaluation module calculates the coupling evaluation results of the calculation and storage.

3. The evaluation method for coupling according to claim 2, wherein The to-be-tested system environment described in Step 1 means that through network communication, the user interaction module and the evaluation module are initialized and deployed on the coupling evaluation device and started; through network transmission, the application load is installed and deployed in the to-be-tested system, and through network communication, the data collection and storage module is installed and deployed on the to-be-tested system.

4. The evaluation method for coupling according to claim 2, characterized in that, The system-level calculation performance index and storage performance index described in Step 2 include system-level indexes of system CPU, system memory, and system disk.

5. The evaluation method for coupling according to claim 2, wherein The process-level calculation performance index and storage performance index described in Step 2 include process-level indexes of process CPU, process memory, and process disk I / O.

6. The evaluation method for the coupling of application load computing performance and storage performance according to claim 2, wherein, The user interaction module receiving the load startup command parameters, encapsulating them into a JSON file, and sending them as described in Step 3 means that the user interaction module will receive the application load startup command parameters input by the user, including the application load startup command parameter verification unit and the connection sending unit. The application load startup command parameter verification unit receives the application load startup command parameters input by the user and performs verification. The connection sending unit uses the to-be-tested system information input by the user to connect to the to-be-tested system, and through network communication, encapsulates the application load startup command parameters into a JSON file and sends it to the application load execution module of the to-be-tested system.

7. The evaluation method for the coupling of application load computing performance and storage performance according to claim 2, wherein, The steps for the application load execution module to execute the application load program and send an index collection instruction described in Step 4 are as follows: First step: The application load execution module splits the application startup parameters in the JSON file according to the preset delimiter. Second step: All the application load startup parameters obtained by splitting are composed into an executable Shell test script. Third step: The application load program is executed through the Shell test script, and an index collection instruction is sent to the data collection and storage module while the application load program is being executed.

8. The evaluation method for coupling according to claim 7, wherein The preset delimiter is set according to the JSON parameter format, using the pipe symbol "|" as the first-level delimiter and the semicolon ";" as the second-level delimiter.

9. The evaluation method for coupling according to claim 2, characterized in that The collection of index data described in Step 5 means that after the data collection and storage module receives the index collection instruction sent by the collection and storage module, it collects the indexes of the calculation performance and storage performance in real time during the execution of the load program and persists and stores them in the time series database.

10. The evaluation method for coupling according to claim 2, characterized in that, The steps for obtaining the coupling evaluation results of the calculation performance and storage performance described in Step 6 are as follows: First step: The evaluation module reads the collected system-level calculation performance and storage performance indexes and process-level calculation performance and storage performance indexes from the time series database, aligns the collected data by timestamp or event, and data alignment means aligning the monitoring data from different time periods and different sources at the same time point. In the second step, perform statistical analysis on the aligned data, standardize the timestamps, and standardize the timestamp formats and data precisions of each data source. In the third step, address the issue of inconsistent collection frequencies among different data sources. Through interpolation and resampling techniques, align the data at unified time points. In the third step, calculate the correlation coefficient matrix among various performance indicators in the collected data to identify pairs of indicators with collinearity and redundant pairs of indicators with high correlation. For the identified redundant indicators, perform reasonable processing through principal component analysis (PCA) and feature selection. In the fourth step, calculate the coupling degree between the overall computing performance and storage performance. Use the entropy method to calculate the index weights of the computing performance indicators and storage performance indicators respectively, use the comprehensive evaluation model to calculate the comprehensive evaluation index of the computing performance indicators and storage performance indicators, and use the coupling degree model to calculate the coupling relationship between the performance and storage performance. In the fifth step, generate a report based on the results of the coupling analysis and display it through the user interaction module.

11. The evaluation method for coupling according to claim 10, wherein The calculation of the index weights of the computing performance indicators and storage performance indicators respectively using the entropy method is obtained through the following steps: In the first step, calculate the degree of variability of each indicator: Among them, P ij represents the proportion of the value of the j-th index in the i-th group of samples, and x ij represents the value of the j-th index after data alignment and correlation analysis of the i-th group of samples. n represents the total number of sample groups in computing performance or storage performance, and m represents the total number of indices included in each sample; In the second step, calculate the information entropy value of each indicator: Among them, E j represents the information entropy value of the j-th index, ln represents the logarithmic operation with the natural constant e as the base, and ∑· represents the summation operation; In the third step, calculate the weight of each indicator: Among them, W j represents the weight of the j-th index.

12. The evaluation method of coupling according to claim 11, characterized in that The comprehensive evaluation index is obtained by the following formula: Among them, represents the comprehensive evaluation index of the i-th group of samples in the k-th performance, where k = 1, 2, u1 represents the comprehensive evaluation index of the computing performance, and u2 represents the comprehensive evaluation index of the storage performance.

13. The evaluation method for coupling according to claim 12, wherein The coupling relationship between the computing performance and storage performance is obtained by the following formula: where C represents the coupling degree of the computing performance and storage performance, with a value range of [0, 1]. The closer the C value is to 1, the stronger the correlation between the computing performance and storage performance and the more ordered it tends to be. Conversely, the weaker the correlation between the computing performance and storage performance and the more disordered it is considered.

Images