PaaS platform resource utilization evaluation method and system

Abstract

The invention relates to the technical field of cloud platforms, in particular to a PaaS platform resource utilization evaluation method and system, and the method comprises the steps: carrying out the processing of collected resource use parameter data, and obtaining a target feature vector; constructing an evaluation model through the target feature vector; and performing evaluation through the evaluation model, and generating an evaluation report. The method has the beneficial effects that the resource utilization efficiency of the PaaS platform service instance can be effectively predicted by constructing the evaluation model, and a scientific method is provided to optimize resource configuration, reduce the operation cost and improve the platform performance by accurately evaluating the model performance. Besides, through visualization of the residual image, deviation of model prediction can be visually recognized, resource management decision is further guided, and therefore more refined resource utilization and cost control are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of cloud platforms, and in particular to a PaaS platform resource utilization evaluation method and system. Background Art

[0002] With the rapid development of cloud computing technology, PaaS platform has become an important choice for enterprises to build and deploy service implementation programs. PaaS platform provides a flexible environment that enables developers to focus on the development of service implementation programs without having to worry about the management and maintenance of the underlying infrastructure. However, the resource utilization of PaaS platform is one of the key indicators to measure its performance and cost-effectiveness. High resource utilization can improve the performance of the platform and reduce operating costs, while low resource utilization may lead to resource waste and increased costs.

[0003] Traditional PaaS platforms usually adopt a static resource allocation strategy, that is, pre-allocating fixed computing, storage and network resources when the service implementation program is deployed; this static resource allocation method cannot effectively respond to the rapid changes in resource demand and easily leads to resource waste or shortage. At the same time, the existing evaluation methods only focus on a single service implementation program or a single resource type, lacking a comprehensive evaluation of the resource utilization of the entire platform, affecting the comprehensiveness of the evaluation of the resource utilization efficiency of the PaaS platform. Summary of the invention

[0004] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0005] In view of the above existing problems, the present invention is proposed.

[0006] Therefore, the present invention provides a PaaS platform resource utilization evaluation method and system, which can solve the problems mentioned in the background technology.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0008] In a first aspect, the present invention provides a PaaS platform resource utilization evaluation method, comprising processing collected resource usage parameter data to obtain a target feature vector;

[0009] Construct an evaluation model through the target feature vector;

[0010] The evaluation is performed through the evaluation model and an evaluation report is generated.

[0011] As a preferred solution of the PaaS platform resource utilization evaluation method of the present invention, the resource usage parameter data collected includes:

[0012] Resource usage parameter data of each service instance collected from the PaaS platform;

[0013] Resource usage parameter data of each service instance collected from the PaaS platform, including

[0014] Use data collection scripts to collect CPU utilization and memory usage of each service instance on the PaaS platform according to preset rules.

[0015] As a preferred solution of the PaaS platform resource utilization evaluation method of the present invention, the collected resource usage parameter data is processed, including:

[0016] The collected resource usage parameter data is cleaned and normalized.

[0017] As a preferred solution of the PaaS platform resource utilization evaluation method of the present invention, wherein: the evaluation model is constructed by the target feature vector, including

[0018] Constructing a first eigenvector matrix and a second eigenvector matrix;

[0019] Projecting the first eigenvector matrix onto the second eigenvector matrix to obtain a target eigenvector matrix after dimension reduction;

[0020] Construct a target evaluation model based on the target feature vector matrix.

[0021] As a preferred solution of the PaaS platform resource utilization evaluation method of the present invention, wherein: constructing a second eigenvector matrix, including

[0022] Perform principal component analysis on the first eigenvector matrix to obtain eigenvalues ​​and corresponding eigenvectors;

[0023] The first k largest eigenvalues ​​and their corresponding eigenvectors are selected to construct a second eigenvector matrix, wherein k is selected so that the variance ratio explained by the first k principal components reaches a preset threshold.

[0024] As a preferred solution of the PaaS platform resource utilization evaluation method of the present invention, wherein: constructing the evaluation model also includes using the method of minimizing the mean square error to train the evaluation model.

[0025] As a preferred solution of the PaaS platform resource utilization evaluation method of the present invention, wherein: CPU utilization includes average CPU utilization and peak CPU utilization;

[0026] Memory usage includes average memory usage and peak memory usage.

[0027] In a second aspect, the present invention provides a PaaS platform resource utilization evaluation system, comprising: a processing module, for processing the collected resource usage parameter data to obtain a target feature vector;

[0028] A building module for building an evaluation model through a target feature vector;

[0029] The generation module is used to evaluate the model and generate an evaluation report.

[0030] In a third aspect, the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0031] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above method when the computer program is executed by a processor.

[0032] Compared with the prior art, the present invention has the following beneficial effects: by constructing an evaluation model, it can effectively predict the resource utilization efficiency of PaaS platform service instances, and by accurately evaluating the model performance, it provides a scientific method to optimize resource allocation, reduce operating costs, and improve platform performance. In addition, through the visualization of the residual graph, the deviation of the model prediction can be intuitively identified, which can further guide resource management decisions, thereby achieving more refined resource utilization and cost control. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0034] Figure 1 The flowchart of the resource utilization evaluation method of PaaS platform is shown in FIG.

[0035] Figure 2 A schematic diagram of the internal structure of a computer device. DETAILED DESCRIPTION

[0036] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0037] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0038] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0039] Example 1

[0040] Reference Figure 1 , which is the first embodiment of the present invention, and provides a PaaS platform resource utilization evaluation method, which includes:

[0041] S1. Process the collected resource usage parameter data to obtain a target feature vector.

[0042] Furthermore, the resource usage parameter data collected includes

[0043] Resource usage parameter data of each service instance collected from the PaaS platform, wherein the resource usage parameter data of each service instance is collected by connecting to an API interface provided by the PaaS platform;

[0044] Resource usage parameter data of each service instance collected from the PaaS platform, including

[0045] Use data collection scripts to collect CPU utilization and memory usage of each service instance on the PaaS platform according to preset rules.

[0046] It should be noted that the preset rule includes a set collection time period, and the generally set time period may be one hour.

[0047] Preferably, using two types of parameters, namely CPU utilization and memory usage in the PaaS platform, as evaluation features can enable evaluation of PaaS platform resource utilization from two dimensions, thereby improving the comprehensiveness of evaluation of PaaS platform resource utilization efficiency and making evaluation results more accurate.

[0048] Furthermore, the CPU utilization includes the average CPU utilization and the peak CPU utilization;

[0049] Memory usage includes average memory usage and peak memory usage.

[0050] It should be noted that the average CPU utilization rate is the average ratio of CPU usage of a service instance within a certain period of time; the peak CPU utilization rate is the highest CPU usage rate reached by a service instance within a certain period of time; the average memory usage is the average amount of memory used by a service instance within a certain period of time; and the peak memory usage is the highest memory usage reached by a service instance within a certain period of time.

[0051] Furthermore, the collected resource usage parameter data is processed, including

[0052] The collected resource usage parameter data is cleaned and normalized.

[0053] It should be noted that data cleaning is used to process missing values, outliers and duplicate values ​​of resource usage parameter data to make the obtained resource usage parameter data more accurate; data normalization refers to the use of a normalization formula to convert resource usage parameter data of different dimensions to the same dimension, which can scale the obtained resource usage parameter data to a specific range (generally 0-1) so that the evaluation model can better process the data. The normalization formula is:

[0054]

[0055] Where x is the original data value; min(x) is the minimum value in the data set; max(x) is the maximum value in the data set; and x′ is the normalized data value.

[0056] Preferably, the collected resource usage parameter data is processed to form a target feature vector related to resource utilization, which can significantly improve the accuracy and efficiency of the resource utilization efficiency evaluation of the PaaS platform, optimize resource allocation, reduce operating costs, and improve the performance and reliability of the platform.

[0057] S2. Construct an evaluation model through the target feature vector.

[0058] Furthermore, the evaluation model is constructed through the target feature vector, including

[0059] Constructing a first eigenvector matrix and a second eigenvector matrix;

[0060] It should be noted that the characteristic values ​​(CPU utilization, peak CPU utilization, average memory usage and peak memory usage) of each service embodiment are organized into a characteristic vector to form a first characteristic vector matrix.

[0061] Projecting the first eigenvector matrix onto the second eigenvector matrix to obtain a target eigenvector matrix after dimension reduction;

[0062] Construct a target evaluation model based on the target feature vector matrix.

[0063] It should be noted that the target evaluation model is specifically:

[0064] y=b0+b1×x1+b2×x2+b3×x3+b4×x4+ε

[0065] Where y is the resource utilization efficiency of the PaaS platform; x1, x2, x3 and x4 represent the average CPU utilization, peak CPU utilization, average memory usage and peak memory usage respectively; b0 is the intercept term; b1, b2, b3 and b4 are the coefficients of each feature; ε is the error term.

[0066] Furthermore, the second eigenvector matrix is ​​constructed, including

[0067] Perform principal component analysis on the first eigenvector matrix to obtain eigenvalues ​​and corresponding eigenvectors;

[0068] It should be noted that principal component analysis (PCA) is performed on the first eigenvector matrix, the covariance matrix is ​​calculated, and eigenvalue decomposition is performed to obtain eigenvalues ​​and corresponding eigenvectors.

[0069] The first k largest eigenvalues ​​and their corresponding eigenvectors are selected to construct a second eigenvector matrix, wherein k is selected so that the variance ratio explained by the first k principal components reaches a preset threshold.

[0070] It should be noted that the specific steps for selecting k are:

[0071] First, calculate the variance contribution rate of each eigenvalue. The variance contribution rate calculation formula is:

[0072]

[0073] In the formula, λ i is the i-th eigenvalue, m is the total number of characteristics;

[0074] Next, calculate the cumulative variance contribution rate. The calculation formula for the cumulative variance contribution rate is:

[0075]

[0076] Finally, the smallest k is selected so that the cumulative variance contribution rate reaches a preset threshold value. In this embodiment, the preset threshold value is 95%.

[0077] Furthermore, constructing the evaluation model also includes training the evaluation model by minimizing the mean square error.

[0078] It should be noted that the optimal coefficient of each feature is found by adopting the minimum mean square error training evaluation model, where the formula for minimizing the mean square error training evaluation model is:

[0079]

[0080] In the formula, y i is the true value; is the predicted value; n is the number of samples

[0081] S3. Evaluate through the evaluation model and generate an evaluation report.

[0082] It should be noted that the evaluation report can be displayed visually in the form of reports or charts, which makes it easy to intuitively understand the evaluation results of PaaS platform resource utilization. Corresponding optimization strategies can be provided based on the evaluation results, which will help managers improve the resource utilization of PaaS platform. For example, the utilization of PaaS platform resources can be optimized by increasing or decreasing resources, adjusting resource allocation, and other strategies.

[0083] In summary, the beneficial effect of the PaaS platform resource utilization evaluation method of the present invention is that by constructing an evaluation model, it can effectively predict the resource utilization efficiency of the PaaS platform service instance, and through accurate evaluation of the model performance, it provides a scientific method to optimize resource allocation, reduce operating costs, and improve platform performance.

[0084] Example 2

[0085] This embodiment provides a method for evaluating the resource utilization of the PaaS platform using a linear regression model in machine learning. Specifically, this script performs the following steps:

[0086] 1. Import necessary libraries:

[0087] NumPy and Pandas are used for data processing.

[0088] train_test_split in sklearn.model_selection is used to divide the dataset.

[0089] LinearRegression in sklearn.linear_model is used to create a linear regression model.

[0090] mean_squared_error, mean_absolute_error, and r2_score from sklearn.metrics are used to evaluate model performance.

[0091] matplotlib.pyplot is used to plot the residual plots.

[0092] 2. Generate sample data:

[0093] Use numpy to generate random data and simulate the average CPU utilization, peak CPU utilization, average memory usage, and peak memory usage of service instances on the PaaS platform.

[0094] From these feature values ​​a target variable (in this case the server response time) is generated which is a weighted sum of the feature values ​​plus some random noise.

[0095] 3. Create DataFrame:

[0096] Combine the feature values ​​and target variable into a DataFrame using pandas.

[0097] 4. Divide the training set and test set:

[0098] Use train_test_split to divide the dataset into training and test sets.

[0099] 5. Training linear regression model:

[0100] Create a LinearRegression model and train it with the training set data.

[0101] 6. Prediction:

[0102] Use the trained model to make predictions on the training and test sets.

[0103] 7. Evaluation Model:

[0104] Use mean_squared_error, mean_absolute_error, and r2_score to calculate the performance indicators of the model on the training set and test set, and print them out.

[0105] 8. Draw the residual graph:

[0106] Use matplotlib to plot the residuals to show the difference between the model's predicted values ​​and the actual values.

[0107] In summary, this is a complete example of a machine learning process, from data preparation to model training, evaluation, and visualization, which is used to show how to use a linear regression model to evaluate the resource utilization efficiency of a PaaS platform.

[0108] Example 3

[0109] This embodiment provides a PaaS platform resource utilization evaluation system, which includes a processing module for processing the collected resource usage parameter data to obtain a target feature vector;

[0110] A building module for building an evaluation model through a target feature vector;

[0111] The generation module is used to evaluate the model and generate an evaluation report.

[0112] The above-mentioned unit modules may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to the above-mentioned modules.

[0113] Example 4

[0114] This embodiment provides a computer device, which may be a terminal, and its internal structure diagram may be as follows: Figure 2 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a PaaS platform resource utilization evaluation method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0115] This embodiment also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it implements: processing the collected resource usage parameter data to obtain a target feature vector; constructing an evaluation model through the target feature vector; performing evaluation through the evaluation model, and generating an evaluation report.

[0116] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A PaaS platform resource utilization evaluation method, characterized in that: include, Process the collected resource usage parameter data to obtain a target feature vector; Construct an evaluation model through the target feature vector; The evaluation is performed using the evaluation model and an evaluation report is generated.

2. The PaaS platform resource utilization evaluation method according to claim 1, characterized in that: The resource usage parameter data collected includes: Resource usage parameter data of each service instance collected from the PaaS platform; The resource usage parameter data of each service instance collected from the PaaS platform includes: Use data collection scripts to collect CPU utilization and memory usage of each service instance on the PaaS platform according to preset rules.

3. The PaaS platform resource utilization evaluation method according to claim 2, characterized in that: The processing of the collected resource usage parameter data includes: The collected resource usage parameter data is cleaned and normalized.

4. The PaaS platform resource utilization evaluation method according to claim 3, characterized in that: The evaluation model is constructed by using the target feature vector, including Constructing a first eigenvector matrix and a second eigenvector matrix; Projecting the first eigenvector matrix onto the second eigenvector matrix to obtain a target eigenvector matrix after dimensionality reduction; A target evaluation model is constructed according to the target feature vector matrix.

5. The PaaS platform resource utilization evaluation method according to claim 4, characterized in that: The second eigenvector matrix is ​​constructed, including Performing principal component analysis on the first eigenvector matrix to obtain eigenvalues ​​and corresponding eigenvectors; The first k largest eigenvalues ​​and their corresponding eigenvectors are selected to construct the second eigenvector matrix, wherein k is selected so that the variance ratio explained by the first k principal components reaches a preset threshold.

6. The PaaS platform resource utilization evaluation method according to claim 5, characterized in that: The constructing of the evaluation model also includes training the evaluation model by minimizing the mean square error.

7. The PaaS platform resource utilization evaluation method according to any one of claims 2 to 6, characterized in that: The CPU utilization includes average CPU utilization and peak CPU utilization; The memory usage includes average memory usage and peak memory usage.

8. A PaaS platform resource utilization evaluation system, characterized in that: include: A processing module is used to process the collected resource usage parameter data to obtain a target feature vector; A building module for building an evaluation model through a target feature vector; The generation module is used to perform evaluation through the evaluation model and generate an evaluation report.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images