Performance prediction method, device, system, computer device and readable storage medium

Abstract

The application provides a performance prediction method, device, system, computer equipment and readable storage medium. The method comprises receiving an original data set recorded in a system service pressure test process sent by a front end, wherein the original data set comprises resource parameters and performance parameters; calling an AI model to perform data profiling on the original data set to generate a first analysis report; screening a training data set from the original data set according to the first analysis report; training a preset initial model by using the training data set to obtain a prediction model; calling a model explanation tool to explain the prediction model to obtain an explanation result; calling the AI model to analyze the explanation result to generate a second analysis report; receiving a system performance prediction request sent by the front end, wherein the system performance prediction request comprises prediction data; and inputting the prediction data into the prediction model to perform system performance prediction. Through the application, the convenience of system performance prediction can be improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a performance prediction method, apparatus, system, computer device, and readable storage medium. Background Technology

[0002] System performance evaluation is primarily used to assess or predict a system's carrying capacity, that is, to evaluate how much business volume the current system can handle, or how many resources the system needs to support a certain business volume. In existing technologies, performance evaluation is required in the following two scenarios: 1) For newly deployed systems, estimating the required resources based on the customer's business volume; 2) For existing systems with resource mismatches, such as low resource utilization leading to significant resource waste, excessive system pressure and slow response, or system instability and frequent failures, estimating the appropriate business volume the system can handle.

[0003] For the first scenario mentioned above, the typical estimation method is as follows: architects and operations personnel estimate the required resources based on contract input and experience, and then deploy a new system. Performance testing is then conducted on the new system, and resources are iteratively adjusted during manual load testing to ultimately derive a suitable resource configuration list and correct the system's configuration.

[0004] For the second scenario mentioned above, the usual estimation method is to manually perform performance stress testing on the deployed system to assess the current business volume that the system can support, and then adjust the system resources (increase or decrease the configuration) according to the customer's actual business volume, and then perform performance stress testing again. This process is repeated until the system resources match the customer's business volume.

[0005] The inventors' research revealed that for the newly deployed system, system deployment is required before performance evaluation. However, without a precise resource list before evaluation, there are potential issues when requesting machines: either too many are requested, leading to resource waste, or too few are requested, requiring re-requests and increasing the deployment cycle. Furthermore, manually performing load testing requires multiple iterations to reach the final result—the optimal resource configuration list. This process is time-consuming, labor-intensive, and requires significant investment of environmental and human resources. Simultaneously, due to the high execution cost, manual load testing cannot perform detailed evaluations of resources for every service; it can only select key core services for evaluation. This results in other services not achieving precise resource adaptation, typically leaving sufficient margin and causing resource waste.

[0006] Therefore, improving the ease of system performance prediction has become a pressing technical problem that needs to be solved in this field. Summary of the Invention

[0007] The purpose of this invention is to provide a performance prediction method, apparatus, system, computer device, and readable storage medium to solve the aforementioned technical problems in the prior art.

[0008] On the one hand, in order to achieve the above objectives, the present invention provides a performance prediction method.

[0009] The performance prediction method includes: receiving a raw dataset recorded during system service load testing sent by the front end, wherein the raw dataset includes resource parameters and performance parameters; calling an AI model to create a data profile of the raw dataset and generating a first analysis report; selecting a training dataset from the raw dataset based on the first analysis report; training a preset initial model using the training dataset to obtain a prediction model; calling a model interpretation tool to interpret the prediction model and obtain an interpretation result; calling the AI ​​model to analyze the interpretation result and generate a second analysis report; receiving a system performance prediction request sent by the front end, wherein the system performance prediction request includes prediction data; and inputting the prediction data into the prediction model to perform system performance prediction.

[0010] Furthermore, the step of calling the AI ​​model to create a data profile of the original dataset and generate a first analysis report includes: calculating the statistics of the resource parameters and / or the performance parameters; generating a statistical chart corresponding to the statistics; calling the AI ​​model to analyze the statistical chart and generate the first analysis report.

[0011] Further, the resource parameters include CPU utilization, number of CPU cores, memory utilization, and memory size; the performance parameters include the number of load testing threads and service throughput. The steps for calculating the statistics of the resource parameters and / or the performance parameters include: calculating the trend relationship between each of the four parameters (CPU utilization, number of CPU cores, memory utilization, and memory size) and the number of load testing threads during a single load test; calculating the cumulative probability of the service throughput during a single load test; calculating the correlation coefficient between each of the four parameters (CPU utilization, number of CPU cores, memory utilization, and memory size) and the service throughput; and calculating the correlation coefficient between the CPU utilization of different services.

[0012] Furthermore, the steps of calling the model interpretation tool to interpret the prediction model and obtain the interpretation results include: randomly extracting data from the training dataset to form samples to be interpreted; calling the SHAP tool to load the prediction model and the samples to be interpreted; and calculating the direction and magnitude of the influence of the input features of the prediction model on the prediction results.

[0013] Furthermore, the initial model is a GradientBoostingRegressor ensemble tree. The steps of training the preset initial model using the training dataset to obtain the prediction model include: calling an automated search and tuning tool; setting the number of trees, learning rate, maximum depth, minimum number of split samples, and downsampling ratio as tuning parameters; selecting a combination of the tuning parameters using the tuning tool; and training the initial model using the training dataset.

[0014] Furthermore, the prediction models include a service throughput prediction model, a CPU core count prediction model, a memory size prediction model, and a pressure-generating thread count prediction model. The input data for the service throughput prediction model includes the number of pressure-generating threads, CPU utilization, average response time, memory utilization, number of CPU cores, and memory size; the output data for the service throughput prediction model includes service throughput. The input data for the CPU core count prediction model includes the number of pressure-generating threads, CPU utilization, average response time, memory utilization, memory size, and service throughput; the output data for the CPU core count prediction model includes the number of CPU cores. The input data for the memory size prediction model includes the number of pressure-generating threads, CPU utilization, average response time, memory utilization, number of CPU cores, and service throughput; the output data for the memory size prediction model includes memory size. The input data for the pressure-generating thread count prediction model includes memory size, CPU utilization, average response time, memory utilization, number of CPU cores, and service throughput; the output data for the pressure-generating thread count prediction model includes the number of pressure-generating threads.

[0015] On the other hand, in order to achieve the above objectives, the present invention provides a system performance prediction device.

[0016] The system performance prediction device includes: a first receiving module for receiving raw datasets recorded during system service stress testing sent by the front end, wherein the raw datasets include resource parameters and performance parameters; a first calling module for calling an AI model to perform data profiling on the raw datasets and generate a first analysis report; a processing module for filtering training datasets from the raw datasets based on the first analysis report; a training module for training a preset initial model using the training datasets to obtain a prediction model; a second calling module for calling a model interpretation tool to interpret the prediction model and obtain interpretation results; a third calling module for calling the AI ​​model to analyze the interpretation results and generate a second analysis report; a second receiving module for receiving system performance prediction requests sent by the front end, wherein the system performance prediction requests include prediction data; and a prediction module for inputting the prediction data into the prediction model to perform system performance prediction.

[0017] On the other hand, in order to achieve the above objectives, the present invention provides a system performance prediction system.

[0018] The performance prediction system includes a user frontend and a service backend. The user frontend receives raw datasets recorded during system service load testing, input by the user. These raw datasets include resource parameters and performance parameters. The service backend invokes an AI model to create a data profile of the raw dataset and generate a first analysis report. The user frontend also displays the first analysis report. Furthermore, the service backend selects a training dataset from the raw dataset based on the first analysis report, trains a preset initial model using the training dataset to obtain a prediction model, invokes a model interpretation tool to interpret the prediction model, obtains an interpretation result, and invokes the AI ​​model to analyze the interpretation result, generating a second analysis report. The user frontend also displays the second analysis report. The user frontend also receives system performance prediction requests from users, including prediction data. Finally, the service backend inputs the prediction data into the prediction model for system performance prediction.

[0019] On the other hand, to achieve the above objectives, the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above method.

[0020] On the other hand, to achieve the above objectives, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above method.

[0021] This invention provides a performance prediction method, apparatus, system, computer device, and readable storage medium. The front end interacts with the user, receiving user-provided data and displaying analysis reports. The back end trains and generates a prediction model based on the data received from the front end. Before training, it calls an AI model to create a data profile of the original dataset, generating a first analysis report to facilitate user data understanding. After training, it calls a model interpretation tool to interpret the prediction model, obtaining interpretation results. The AI ​​model then analyzes these interpretation results, generating a second analysis report, enabling the user to understand the model's prediction logic. Simultaneously, when a user sends a system performance prediction request via the front end, the back end can input prediction data into the prediction model and output prediction results, thus achieving system performance prediction. This invention constructs a complete closed-loop process from data acquisition, AI analysis, model training to performance prediction. Users can conveniently complete system performance evaluation through the front end without needing modeling capabilities or repeated stress testing, significantly reducing the barrier to entry and testing costs. Meanwhile, to address the "black box" problem of traditional prediction models, an AI model is introduced to create data profiles before model training, and a model interpretation tool is introduced after training to perform causal analysis on the prediction process. The AI ​​model is then used to generate structured natural language analysis reports from the interpretation results, enabling users to understand the prediction logic, identify key influencing factors, and achieve interpretable system performance predictions. This improves the usability, credibility, and intelligence level of system performance evaluation. This invention provides performance and resource prediction. Without actual stress testing, users can input relevant data to obtain recommended resource data and accurate performance data, which is impossible with traditional performance testing methods. This significantly reduces labor costs; it greatly lowers the barrier to AI modeling, allowing business personnel to complete model training and prediction without mastering algorithm details; and it provides parameter references for real performance testing, reducing the number of stress test rounds, accelerating the performance testing process, and improving performance testing efficiency. Attached Figure Description

[0022] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A flowchart of the performance prediction method provided in Embodiment 1 of the present invention; Figure 2 A schematic diagram of the performance prediction method provided in the embodiments of the present invention. Figure 3 This is a block diagram of the performance prediction device provided in Embodiment 2 of the present invention; Figure 4 This is a hardware structure diagram of a computer device provided in Embodiment 4 of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0024] Example 1 This invention provides a performance prediction method applied to a performance prediction system. The performance prediction system predicts the matching relationship between the carrying capacity and resources of a service system. The performance prediction system includes a user front-end and a service back-end. The front-end interacts with the user, while the service back-end trains a model and predicts system performance based on the trained model. The service back-end is the execution entity of this method, thus realizing an automated system performance prediction method. Specifically, Figure 1 The flowchart of the performance prediction method provided in Embodiment 1 of the present invention is as follows: Figure 1 As shown, the performance prediction method provided in this embodiment includes the following steps S101 to S108.

[0025] Step S101: Receive the raw dataset recorded during the system service load test sent by the front end.

[0026] The original dataset includes resource parameters and performance parameters.

[0027] Optionally, during the load testing process of the existing service system, resource parameters and performance parameters are collected in real time. Resource parameters reflect the system's resource usage, while performance parameters reflect the system's service responsiveness. Optionally, resource parameters include memory data (e.g., memory utilization, memory size) and CPU data (e.g., CPU utilization, number of CPU cores), while performance parameters include key metrics such as throughput data (tps), thread data (e.g., number of threads applying load), and response time (e.g., 90th, 95th, and 99th percentile average response time, average response time). Data sources can include log systems, monitoring systems, or load testing tools.

[0028] When a user needs to make predictions about a service system based on the performance prediction system, they can upload the original dataset of historical records to the front end, and the service back end receives the original dataset through the front end.

[0029] Optionally, upon receiving the original dataset, it can be saved as a data file in a predefined format. First, metadata abstraction is performed, identifying the field categories of various data types in the original dataset and dividing each category into time, numerical, and categorical fields. Then, data cleaning is performed, converting feature data into numerical variables and standardizing the time format. Specifically, memory size is converted to a numerical variable (1024K to 1024), and time is standardized to %Y-%M-%D %H:%M:%S. Finally, data is aligned according to the time field. Specifically, if the time difference is less than 10 seconds, the nearest data record is selected and concatenated. If no data record exists within a 10-second time difference, the corresponding field is set to empty.

[0030] Step S102: Call the AI ​​model to create a data profile of the original dataset and generate the first analysis report.

[0031] Data profiling refers to the method of understanding data structure and patterns through statistics and modeling. In this step, AI models are invoked to create data profiles. Using AI models (such as large language models or statistical learning models), the raw datasets recorded during system service load testing are analyzed. Statistical and correlation analyses are performed on resource and performance parameters to reveal the intrinsic relationship between resource indicators and performance during system operation. System behavior patterns are extracted from this data, providing a reference for the selection of subsequent training data.

[0032] Optionally, the original dataset, after undergoing the aforementioned metadata abstraction, data cleaning, and data alignment processes, is then used to call an AI model for data profiling. During data profiling and report generation, statistical relationships between different variables can be analyzed, images can be generated using a pre-defined graph analysis library, and the image content or correlation values ​​can be analyzed to generate textual interpretations. For example, the first analysis report may include trend charts, distribution charts, correlation charts, explanatory text for each chart, and a summary diagnostic recommendation paragraph.

[0033] After the service backend generates the first analysis report, it is sent to the frontend for display. The analysis results are presented to users through a graphical interface or web visualization tools, improving the interpretability and interactivity of the data and making it easier for users to understand the data.

[0034] Step S103: Select training datasets from the original datasets based on the first analysis report.

[0035] The first analysis report can help the system automatically identify key feature fields, such as selecting data with high correlation or stable trends as model input, thereby constructing a high-quality and representative subset of data as a training dataset.

[0036] Step S104: Train the preset initial model using the training dataset to obtain the prediction model.

[0037] In this step, a pre-set machine learning model, such as an ensemble learning regression model, is used as the initial model; input features and target variables are extracted from the training dataset, the input features are used as the input data of the initial model, and the target variables are used as the prediction results of the initial model. The model iteratively fits the optimal function to finally obtain the prediction model.

[0038] Optionally, the prediction model includes a service throughput prediction model, a CPU core count prediction model, a memory size prediction model, and a pressure-generating thread count prediction model. By combining these four models, bidirectional prediction of resource allocation and target performance is achieved, adapting to diverse prediction needs and improving the versatility of the prediction tool.

[0039] The input data for the service throughput prediction model includes the number of threads under load, CPU utilization, average response time, memory utilization, number of CPU cores, and memory size. The output data includes service throughput. This service throughput prediction model can predict the maximum throughput capacity of a system given a set of configuration parameters, quickly obtaining the load ceiling of the system under test without actual load testing, and identifying configuration waste or bottleneck risks.

[0040] The CPU core count prediction model takes into account the number of threads under load, CPU utilization, average response time, memory utilization, memory size, and service throughput as input data, and outputs the number of CPU cores. This model predicts the number of CPU cores needed to support a given target workload (such as service throughput) and other supporting conditions, thereby enabling on-demand allocation of resources, avoiding excessive redundancy, and allowing hardware resources to be derived from performance metrics, thus improving resource utilization.

[0041] The input data for the memory size prediction model includes the number of threads under pressure, CPU utilization, average response time, memory utilization, number of CPU cores, and service throughput. The output data includes memory size. This model can predict the minimum memory configuration required to achieve a target throughput under given concurrency and CPU resources, enabling memory resource planning, determining whether current memory is redundant or overloaded, and automatically optimizing the system's memory allocation strategy to avoid configuration difficulties and achieve refined resource quota management. Combined with the CPU core count prediction model, it can achieve comprehensive configuration balance.

[0042] The input data for the stress-prone thread count prediction model includes memory size, CPU utilization, average response time, memory utilization, number of CPU cores, and service throughput. The output data includes the number of stress-prone threads. This model predicts the appropriate number of stress-prone threads (i.e., concurrency) to achieve maximum throughput efficiency given available resources (CPU / memory, etc.) and a performance target (throughput). It can be applied to load testing script tuning, concurrency parameter optimization during performance testing, and identifying performance degradation caused by excessive concurrency. This improves load testing efficiency and accuracy, reduces test iteration costs, supports modeling the non-linear relationship between thread count and throughput, provides optimal stress-prone strategies for the system, and can be used for automatic recommendation of load testing parameters in test automation platforms.

[0043] Optionally, after obtaining the prediction model, a prediction discrepancy assessment can be performed, comparing the model's predicted data with the actual data, and outputting a distribution map of the actual and predicted data.

[0044] Step S105: Call the model interpretation tool to interpret the prediction model and obtain the interpretation results.

[0045] While machine learning models can make predictions, they are often considered "black box models." In this step, a model interpretation mechanism is introduced by calling a model interpretation tool to deeply analyze the prediction model, showing the impact of each input feature on the final prediction result. This makes the prediction model a visible logical structure, enhancing users' trust and understanding of the prediction model.

[0046] Optionally, in the implementation of this invention, the model interpretation tools invoked include SHAP, LIME, Permutation Importance, Partial Dependency Graph (PDP), XGBoost's built-in TreeExplainer, and counterfactual interpreters (such as DiCE). Depending on the model type and interpretation scenario, a suitable interpretation mechanism can be flexibly selected to improve model understandability, system debugging capabilities, and user trust.

[0047] Step S106: Call the AI ​​model to analyze the interpretation results and generate a second analysis report.

[0048] In this step, the explanation results generated in the previous steps are used as the intermediate semantic layer output. These are further processed by an AI model through language generation, causal reasoning, knowledge enhancement, and suggestion extraction to form a human-oriented, structured analysis report. Specifically, based on the explanation results provided by tools such as SHAP, and combined with the stress testing background, prediction targets, and system status, structured analysis results are generated using natural language processing.

[0049] After the service backend generates the second analysis report, it is sent to the frontend for display. The analysis results are presented to users through a graphical interface or web visualization tools, which improves the interpretability of the model and helps users understand the model's prediction logic, thereby increasing model trust and deployment efficiency.

[0050] Step S107: Receive the system performance prediction request sent by the front end.

[0051] The system performance prediction request includes prediction data.

[0052] After training the initial model with training data to obtain a predictive model, users can use this predictive model as a tool to test system performance, replacing actual performance testing. When testing is needed, a system performance prediction request is submitted through the front-end, uploading the predicted data to the front-end. For example, users can input interface data on the front-end to perform performance prediction.

[0053] Optionally, the backend service framework adopts the Fastapi framework, which is used to process service requests and push data generated during model training, such as the first analysis report and the second analysis report.

[0054] Step S108: Input the prediction data into the prediction model to predict system performance.

[0055] The front end transmits the prediction data to the back end. The back end extracts the input data required by the prediction model from the prediction data and inputs it into the prediction model. The prediction model then outputs the prediction results and provides feedback to the user. For example, when a user wants to predict service throughput, the back end extracts the number of threads under pressure, CPU utilization, average response time, memory utilization, number of CPU cores, and memory size from the prediction data and inputs them into the service throughput prediction model to output the service throughput.

[0056] Optionally, Figure 2 This is a schematic diagram of the performance prediction method provided in an embodiment of the present invention. The backend uses WebSocket to push data to the frontend. During model training, a model effect image is generated and pushed to the frontend for visualization. The current model training progress is also pushed, and the frontend displays the current training progress in real time. A large model is called to generate a real-time analysis report, which is pushed to the frontend for display, helping users understand the current model's performance. Finally, the trained model is deployed. Figure 2 As shown, after a user requests a connection to WebSockets (WS), they upload data files and send an asynchronous request for model training. During training, WS pushes training progress updates in real time and generates model evaluation reports in real time. After model training is complete, the model is deployed, its deployment status is pushed, and the model is explained. Users can download the model evaluation report.

[0057] The performance prediction method provided in this embodiment is applied to a performance prediction system. This system provides a front-end for user interaction, receiving user-provided data and displaying analysis reports. The back-end, based on the data received by the front-end, trains and generates a prediction model. Before training, it calls an AI model to create a data profile of the original dataset, generating a first analysis report to facilitate user understanding. After training, it calls a model interpretation tool to interpret the prediction model, obtaining interpretation results. The AI ​​model then analyzes these interpretation results, generating a second analysis report, enabling users to understand the model's prediction logic. Simultaneously, when a user sends a system performance prediction request via the front-end, the back-end can input prediction data into the prediction model and output prediction results, thus achieving system performance prediction. Using the performance prediction method provided in this embodiment, a complete closed-loop process from data acquisition, AI analysis, model training to performance prediction is constructed. Users can conveniently complete system performance evaluation through the front-end without needing modeling capabilities or repeated load testing, significantly reducing the barrier to entry and testing costs. Meanwhile, to address the "black box" problem of traditional prediction models, an AI model is introduced to create data profiles before model training, and a model interpretation tool is introduced after model training to perform causal analysis on the prediction process. The AI ​​model is then used to generate structured natural language analysis reports on the interpretation results, enabling users to understand the prediction logic, identify key influencing factors, and achieve interpretable system performance prediction. This improves the usability, credibility, and intelligence level of system performance evaluation.

[0058] Optionally, in one embodiment, the step of calling an AI model to create a data profile of the original dataset and generate a first analysis report includes: calculating statistics of resource parameters and / or performance parameters; generating statistical graphs corresponding to the statistics; calling an AI model to analyze the statistical graphs and generate a first analysis report.

[0059] In this embodiment, the system's operating characteristics are revealed through data statistics and graphical visualization of the large amount of raw data collected during the performance stress test (such as CPU utilization, memory utilization, response time, throughput, number of stress-generating threads, etc.). Based on this, an AI model is used to perform semantic analysis to generate a first analysis report that is easy for humans to understand, serving as a data understanding step before model training.

[0060] Specifically, quantitative statistical analysis is performed on key indicators during the stress testing process, including but not limited to: maximum value, minimum value, average value, median, standard deviation; quantiles (90p, 95p, 99p); time series trends; cluster analysis (such as clustering CPU utilization to observe its load distribution); for correlation relationships, correlation coefficients (such as Pearson correlation coefficient), covariance matrices, etc. can also be calculated to extract the feature expression of the original data and provide a reference for the selection of training data before model training.

[0061] Then, based on the above statistics, construct visualization charts, including but not limited to: line charts showing the trend of a certain indicator over time or the number of threads; scatter plots or heatmaps showing the correlation between two parameters (such as CPU utilization vs. throughput); bar charts or box plots showing the distribution of indicators and outliers; cumulative distribution charts showing the cumulative probability performance of indicators such as TPS; or multiple charts linked together, such as a combined chart of response time and CPU fluctuation trends, to present the system behavior pattern in graphical form and assist the AI ​​model in discovering hidden patterns.

[0062] Finally, AI models (such as Wenxin Yiyan, GPT, and Claude) are used to automatically interpret the above graphical content and generate natural language analysis segments, including but not limited to: trend interpretation, such as CPU utilization increasing significantly with the number of threads, showing a linear relationship; bottleneck identification, such as TPS tending to saturate after CPU utilization exceeds 80%, possibly due to limitations in computing resources; resource utilization suggestions, such as memory utilization consistently below 50%, indicating potentially redundant configurations; and correlation discovery, such as a positive correlation between CPU core count and TPS (r=0.78), while memory size has no significant relationship with TPS. This transforms complex graphical data into natural language knowledge, empowering users' data understanding and system cognition.

[0063] The performance prediction method provided in this embodiment introduces a data profiling step before model training. This not only enables the extraction and graphical representation of statistics from the original stress test data, but also automatically analyzes the statistical graphs by calling an AI model, thereby generating a structured and linguistic first analysis report.

[0064] Optionally, in one embodiment, resource parameters include CPU utilization, number of CPU cores, memory utilization, and memory size; performance parameters include the number of load testing threads and service throughput. The steps for calculating the statistics of resource parameters and / or performance parameters include: calculating the trend relationship between each of the four parameters (CPU utilization, number of CPU cores, memory utilization, and memory size) and the number of load testing threads during a single load test; calculating the cumulative probability of service throughput during a single load test; calculating the correlation coefficient between each of the four parameters (CPU utilization, number of CPU cores, memory utilization, and memory size) and service throughput; and calculating the correlation coefficient between the CPU utilization of different services.

[0065] Specifically, one load test is selected, and the trend relationships between CPU utilization, CPU core count, memory utilization, and memory size are calculated. Trend graphs are generated for different variables, and the Wenxin Yiyan large-scale model is used to analyze these trend graphs, generating an analysis report to interpret the correlation trends between variables. Another load test is selected, and the cumulative probability analysis graph of service throughput (TPS) is calculated. The Wenxin Yiyan large-scale model is used to analyze this cumulative probability distribution data, generating an analysis report to interpret the cumulative probability distribution. Pearson correlation coefficients are calculated between different CPU data (CPU utilization and CPU core count), memory data (memory size and memory utilization), and service throughput (TPS). Correlation graphs are plotted and input into the Wenxin Yiyan large-scale model to generate a correlation analysis report. Finally, Pearson correlation coefficients are calculated for different service CPU utilization rates, and the correlation data is input into the Wenxin Yiyan large-scale model to generate a correlation analysis report to interpret the correlations between the parameters.

[0066] The performance prediction method provided in this embodiment constructs a structured profile of the system stress test data by conducting multi-dimensional statistical analysis on the trend relationship, distribution characteristics, and correlation between resource parameters and performance parameters. This reveals the dynamic response relationship between resources and load, provides a basis for model input feature selection, reduces redundant variables, improves prediction accuracy and training efficiency, enriches the content of the first analysis report, making it more structured, interpretable, and instructive, and assists users in making in-depth understandings of training data selection.

[0067] Optionally, in one embodiment, the step of calling a model interpretation tool to interpret the prediction model and obtain the interpretation result includes: randomly extracting data from the training dataset to form samples to be interpreted; calling the SHAP tool to load the prediction model and the samples to be interpreted; and calculating the direction and magnitude of the influence of the input features of the prediction model on the prediction result.

[0068] In this embodiment, the SHAP tool is used to load the prediction model and samples. The trained model is input into the SHAP tool to analyze the impact of different feature data on the dependent variable, calculating the direction (positive or negative) and magnitude of the impact of each input feature on the prediction result. This reveals the internal reasoning mechanism of the model, making the model's predictions more understandable and enhancing user trust. Specifically, several samples are randomly selected from the training dataset for interpretation, avoiding the interpretation of isolated examples. A single sample can be selected for local interpretation, or the average of multiple samples can be used for global interpretation. The SHAP tool outputs the Shap value for each feature in the sample, reflecting the marginal influence of that feature in that sample. A positive value indicates a positive impact, i.e., improving the prediction result, while a negative value indicates a negative impact, i.e., suppressing the prediction result.

[0069] The performance prediction method provided in this embodiment introduces the SHAP tool to interpret the prediction model, which quantifies the specific direction and degree of influence of each input feature on the prediction result. This makes the model's reasoning process clearly visible, avoiding the incomprehensible problems of traditional black-box models. Users can not only understand how the prediction results are derived, but also identify key influencing factors, increasing their trust in the model.

[0070] Optionally, in one embodiment, the initial model is a GradientBoostingRegressor ensemble tree. The steps of training the preset initial model using the training dataset to obtain the prediction model include: calling an automated search and tuning tool; setting the number of trees, learning rate, maximum depth, minimum number of split samples, and downsampling ratio as tuning parameters; selecting a combination of tuning parameters using the tuning tool; and training the initial model using the training dataset.

[0071] Specifically, GradientBoostingRegressor ensemble trees are used for training and prediction. GradientBoostingRegressor is an ensemble learning model based on boosting trees that can capture nonlinear relationships, performs well with small samples, is insensitive to feature scale, and supports feature importance analysis. In this embodiment, GradientBoostingRegressor ensemble trees are used as the initial model, which is suitable for regression tasks in system performance evaluation, such as predicting throughput, number of CPU cores, memory size, etc.

[0072] During training, Bayesian automated search is used for hyperparameter tuning. The BayesSearchCV tool can be used, and the process includes the following steps: Bayesian hyperparameter tuning selects the optimal combination of parameters. For example, the main tuning parameters are: number of trees (50, 500), learning rate (1e-4, 1e-1), maximum depth (3, 10), minimum number of split samples (2, 20), and downsampling ratio (0.5, 1.0). A grid search is employed, traversing all combinations to intelligently search for the optimal solution in the parameter space, thus improving efficiency.

[0073] The performance prediction method provided in this embodiment introduces the GradientBoostingRegressor ensemble tree model and combines it with automated parameter tuning tools to jointly optimize multiple hyperparameters. This enables the automatic selection of the optimal model structure and parameter combination without human intervention, thereby improving the fitting accuracy and generalization ability of the prediction model and providing more reliable and adaptive prediction capabilities for system performance evaluation.

[0074] In summary, this invention introduces a spatiotemporal dimension and combines it with relationships between entities to form a richer feature space, enhancing data mining and analysis capabilities. Interpretability is particularly important when building complex models; interpretability techniques allow users to understand the model's decision-making process, thereby increasing model transparency, enhancing trust, and facilitating debugging and optimization. Continuous Integration and Continuous Deployment (CICD) processes automate the entire model process from training to deployment, significantly improving development efficiency and model update frequency. By integrating multiple models and combining their advantages, more robust prediction results can be obtained. Furthermore, automated parameter tuning optimizes model hyperparameter configuration, further improving model performance and reducing the need for manual intervention.

[0075] Example 2 Corresponding to Embodiment 1 above, Embodiment 2 of the present invention provides a performance prediction device. The technical features and corresponding technical effects can be referred to Embodiment 1 above, and will not be repeated in this embodiment. Figure 3 This is a block diagram of the performance prediction device provided in Embodiment 2 of the present invention, as shown below. Figure 3 As shown, the device includes: a first receiving module 201, a first calling module 202, a processing module 203, a training module 204, a second calling module 205, a third calling module 206, a second receiving module 207, and a prediction module 208.

[0076] The system includes a first receiving module 201 for receiving raw datasets recorded during system service stress testing sent by the front end, wherein the raw datasets include resource parameters and performance parameters; a first calling module 202 for calling an AI model to perform data profiling on the raw datasets and generate a first analysis report; a processing module 2003 for selecting training datasets from the raw datasets based on the first analysis report; a training module 204 for training a preset initial model using the training datasets to obtain a prediction model; a second calling module 205 for calling a model interpretation tool to interpret the prediction model and obtain interpretation results; a third calling module 206 for calling the AI ​​model to analyze the interpretation results and generate a second analysis report; a second receiving module 207 for receiving a system performance prediction request sent by the front end, wherein the system performance prediction request includes prediction data; and a prediction module 208 for inputting the prediction data into the prediction model to perform system performance prediction.

[0077] Optionally, in one embodiment, the first calling module includes: a first calculation unit, configured to calculate statistics of the resource parameters and / or the performance parameters; a generation unit, configured to generate a statistical graph corresponding to the statistics; and a first calling unit, configured to call the AI ​​model to analyze the statistical graph and generate the first analysis report.

[0078] Optionally, in one embodiment, the resource parameters include CPU utilization, number of CPU cores, memory utilization, and memory size, and the performance parameters include the number of load testing threads and service throughput. When the first computing unit calculates the statistics of the resource parameters and / or the performance parameters, the specific steps performed include: calculating the trend relationship between each of the four parameters (CPU utilization, number of CPU cores, memory utilization, and memory size) and the number of load testing threads during a single load test; calculating the cumulative probability of the service throughput during a single load test; calculating the correlation coefficient between each of the four parameters (CPU utilization, number of CPU cores, memory utilization, and memory size) and the service throughput; and calculating the correlation coefficient between the CPU utilization of different services.

[0079] Optionally, in one embodiment, the second calling module includes: an extraction unit for randomly extracting data from the training dataset to form samples to be explained; a second calling unit for calling the SHAP tool to load the prediction model and the samples to be explained; and a second calculation unit for calculating the direction and magnitude of the influence of the input features of the prediction model on the prediction result.

[0080] Optionally, in one embodiment, the initial model is a GradientBoostingRegressor ensemble tree, and the training module includes: a third calling unit for calling an automated search and tuning tool; a setting unit for setting the number of trees, learning rate, maximum depth, minimum number of split samples, and downsampling ratio as tuning parameters; and a training unit for selecting a combination of the tuning parameters through the tuning tool and training the initial model using the training dataset.

[0081] Optionally, in one embodiment, the prediction model includes a service throughput prediction model, a CPU core count prediction model, a memory size prediction model, and a pressure-generating thread count prediction model; the input data of the service throughput prediction model includes the number of pressure-generating threads, CPU utilization, average response time, memory utilization, number of CPU cores, and memory size, and the output data of the service throughput prediction model includes service throughput; the input data of the CPU core count prediction model includes the number of pressure-generating threads, CPU utilization, average response time, memory utilization, memory size, and service throughput, and the output data of the CPU core count prediction model includes the number of CPU cores; the input data of the memory size prediction model includes the number of pressure-generating threads, CPU utilization, average response time, memory utilization, number of CPU cores, and service throughput, and the output data of the memory size prediction model includes memory size; the input data of the pressure-generating thread count prediction model includes memory size, CPU utilization, average response time, memory utilization, number of CPU cores, and service throughput, and the output data of the pressure-generating thread count prediction model includes the number of pressure-generating threads.

[0082] Example 3 Corresponding to Embodiment 1 above, Embodiment 3 of the present invention provides a performance prediction system. The technical features and corresponding effects are detailed in Embodiment 1 above and will not be repeated here. The performance prediction system includes a user front-end and a service back-end, wherein: the user front-end receives raw datasets recorded during system service stress testing input by a user, wherein the raw datasets include resource parameters and performance parameters; the service back-end calls an AI model to perform data profiling on the raw datasets and generates a first analysis report; the user front-end also displays the first analysis report; the service back-end further selects training datasets from the raw datasets based on the first analysis report, trains a preset initial model using the training datasets to obtain a prediction model, calls a model interpretation tool to interpret the prediction model, obtains interpretation results, calls the AI ​​model to analyze the interpretation results, and generates a second analysis report; the user front-end also displays the second analysis report; the user front-end also receives system performance prediction requests from users, wherein the system performance prediction requests include prediction data; and the service back-end further inputs the prediction data into the prediction model for system performance prediction.

[0083] Example 4 This embodiment also provides a computer device, such as a smartphone, tablet computer, laptop computer, desktop computer, rack server, blade server, tower server, or cabinet server (including a standalone server or a server cluster composed of multiple servers), etc., capable of executing programs. Figure 4As shown, the computer device 01 in this embodiment includes, but is not limited to, a memory 012 and a processor 011 that can be interconnected via a system bus, such as... Figure 4 As shown. It should be noted that, Figure 3 Only a computer device 01 with component memory 012 and processor 011 is shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0084] In this embodiment, the memory 012 (i.e., the readable storage medium) includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 012 may be an internal storage unit of the computer device 01, such as the hard disk or memory of the computer device 01. In other embodiments, the memory 012 may also be an external storage device of the computer device 01, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 01. Of course, the memory 012 may include both the internal storage unit and its external storage device of the computer device 01. In this embodiment, the memory 012 is typically used to store the operating system and various reference software installed on the computer device 01, such as the program code of the performance prediction device in Embodiment 2. In addition, memory 012 can also be used to temporarily store various types of data that have been output or will be output.

[0085] In some embodiments, processor 011 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. Processor 011 is typically used to control the overall operation of computer device 01. In this embodiment, processor 011 is used to run program code stored in memory 012 or process data, such as performance prediction methods.

[0086] Example 5 This embodiment also provides a computer-readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, server, app reference store, etc., which stores a computer program. When the program is executed by a processor, it implements the corresponding function. The computer-readable storage medium of this embodiment is used to store a performance prediction device, and when executed by a processor, it implements the performance prediction method of Embodiment 1.

[0087] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0088] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0089] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0090] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A performance prediction method, characterized in that, include: Receive the raw dataset recorded during the system service stress test sent by the front end, wherein the raw dataset includes resource parameters and performance parameters; The AI ​​model is invoked to create a data profile of the original dataset, generating a first analysis report; The training dataset is selected from the original dataset based on the first analysis report; The preset initial model is trained using the training dataset to obtain the prediction model; The prediction model is interpreted using a model interpretation tool to obtain the interpretation results; The AI ​​model is invoked to analyze the interpretation results and generate a second analysis report; Receive a system performance prediction request sent by the front end, wherein the system performance prediction request includes prediction data; and The predicted data is input into the prediction model to predict system performance.

2. The performance prediction method according to claim 1, characterized in that, The steps for calling an AI model to create a data profile of the original dataset and generate a first analysis report include: Calculate the statistics of the resource parameters and / or the performance parameters; Generate the statistical graph corresponding to the statistic; The AI ​​model is invoked to analyze the statistical chart, generating the first analysis report.

3. The performance prediction method according to claim 2, characterized in that, The resource parameters include CPU utilization, number of CPU cores, memory utilization, and memory size; the performance parameters include the number of threads applying load and service throughput; and the steps for calculating the statistics of the resource parameters and / or the performance parameters include: Calculate the trend relationship between each of the four parameters (CPU utilization, number of CPU cores, memory utilization, memory size, and number of threads) and the number of threads running the load test during a single load test. Calculate the cumulative probability of the service throughput during a single load test; Calculate the correlation coefficient between each of the four parameters (CPU utilization, number of CPU cores, memory utilization, and memory size) and the service throughput; Calculate the correlation coefficient between the CPU utilization rates of different services.

4. The performance prediction method according to claim 1, characterized in that, The steps of calling a model interpretation tool to interpret the prediction model and obtain the interpretation results include: Data is randomly drawn from the training dataset to form samples to be interpreted; The SHAP tool is invoked to load the prediction model and the sample to be explained; and Calculate the direction and magnitude of the influence of the input features of the prediction model on the prediction results.

5. The performance prediction method according to claim 1, characterized in that, The initial model is a GradientBoostingRegressor ensemble tree. The steps to train the preset initial model using the training dataset to obtain the prediction model include: Call the automated parameter tuning tool; Set the number of trees, learning rate, maximum depth, minimum number of split samples, and downsampling ratio as tuning parameters; The combined parameters of the parameter tuning parameters are selected using the parameter tuning tool, and the initial model is trained using the training dataset.

6. The performance prediction method according to claim 1, characterized in that, The prediction models include a service throughput prediction model, a CPU core count prediction model, a memory size prediction model, and a pressure-generating thread count prediction model. The input data of the service throughput prediction model includes the number of threads under pressure, CPU utilization, average response time, memory utilization, number of CPU cores, and memory size. The output data of the service throughput prediction model includes service throughput. The input data of the CPU core count prediction model includes the number of threads under pressure, CPU utilization, average response time, memory utilization, memory size and service throughput, and the output data of the CPU core count prediction model includes the number of CPU cores. The input data of the memory size prediction model includes the number of threads under pressure, CPU utilization, average response time, memory utilization, number of CPU cores and service throughput, and the output data of the memory size prediction model includes memory size. The input data of the pressure-generating thread count prediction model includes memory size, CPU utilization, average response time, memory utilization, number of CPU cores, and service throughput. The output data of the pressure-generating thread count prediction model includes the number of pressure-generating threads.

7. A system performance prediction device, characterized in that, include: The first receiving module is used to receive the raw dataset recorded during the system service stress test sent by the front end, wherein the raw dataset includes resource parameters and performance parameters; The first calling module is used to call the AI ​​model to create a data profile of the original dataset and generate a first analysis report; The processing module is used to filter the training dataset from the original dataset according to the first analysis report; The training module is used to train a preset initial model using the training dataset to obtain a prediction model; The second calling module is used to call the model interpretation tool to interpret the prediction model and obtain the interpretation result; The third calling module is used to call the AI ​​model to analyze the interpretation results and generate a second analysis report; The second receiving module is configured to receive a system performance prediction request sent by the front end, wherein the system performance prediction request includes prediction data; and The prediction module is used to input the prediction data into the prediction model to predict system performance.

8. A performance prediction system, characterized in that, This includes the user front-end and service back-end, among which: The user front-end is used to receive raw datasets recorded during system service stress testing, which are input by the user. The raw datasets include resource parameters and performance parameters. The service backend is used to call the AI ​​model to create a data profile of the original dataset and generate a first analysis report; The user front-end is also used to display the first analysis report; The service backend is also used to select a training dataset from the original dataset according to the first analysis report, train a preset initial model using the training dataset to obtain a prediction model, call a model interpretation tool to interpret the prediction model to obtain an interpretation result, call the AI ​​model to analyze the interpretation result, and generate a second analysis report. The user front-end is also used to display the second analysis report; The user front-end is also used to receive system performance prediction requests from users, wherein the system performance prediction request includes prediction data; and The service backend is also used to input the prediction data into the prediction model for system performance prediction.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images