Application clustering-based application performance prediction and fault detection method and system, and storage medium

Abstract

The invention discloses an application performance prediction and fault detection method and system based on application clustering and a storage medium. The method comprises the following steps: collecting system equipment bottom layer data of each node in a network and application performance data and application response time data of an application layer, clustering all collected application performance data, and dividing the application performance data into four classes of CPU-based, memory-based, hard disk-based and network-based; and for all applications of each group of categories, establishing a logistic regression model by using the equipment underlying data, predicting application response time data, training to obtain a prediction model, and for a certain application, predicting the application by using a corresponding prediction model for a group of new equipment underlying data to obtain an application response time prediction result. And constructing a logistic regression dichotomy model for the application response time prediction result and the actual application response time, and training the logistic regression dichotomy model. According to the invention, reliable and effective individual application performance prediction and fault detection can be carried out on each application running on the system.

Description

Technical Field

[0001] The present invention belongs to the field of cloud computing application performance fault detection, and relates to an application performance and fault prediction method, and in particular to an application performance prediction and fault detection method, system and storage medium based on application clustering. Background Art

[0002] In the field of cloud computing technology, application performance prediction and fault detection are key to ensuring system stability and optimizing resource usage. In the industry, mainstream cloud vendors provide application performance fault monitoring and alarm functions. Amazon Web Services (AWS)'s CloudWatch service [1] provides a comprehensive solution for application performance prediction and fault detection. CloudWatch can monitor the performance indicators of AWS resources and applications in real time, such as CPU usage, memory usage, and network traffic. When these indicators exceed the preset threshold, CloudWatch triggers an alarm. IBM Cloud provides IBM Cloud Monitoring [2] and IBM Cloud Log Analysis services, which integrate open source tools such as Prometheus and Grafana to achieve comprehensive performance monitoring and log analysis. IBM CloudMonitoring can monitor the performance indicators of cloud resources and applications in real time, and provide customized alarms and visual reports. Tencent Cloud also performs well in performance monitoring and fault detection. Its Cloud Monitor service provides real-time monitoring and alarm functions for cloud resources, supporting monitoring of multiple indicators such as CPU, memory, and network. At the same time, Tencent Cloud's Log Service (CLS) [3] provides centralized log management and analysis functions to help users quickly locate and solve problems. The cloud monitoring technology provided by the industry is usually not tailored for specific applications and can only alert application anomalies by manually setting thresholds.

[0003] Traditional methods for application performance prediction and fault alarms include monitoring system performance indicators (such as CPU usage, memory usage, etc.) based on preset thresholds, and triggering alarms when the indicators exceed the preset thresholds. The main problems of this approach are: fixed thresholds: difficult to adapt to different workloads and usage scenarios, which may lead to false positives or false negatives; poor flexibility: unable to dynamically adjust thresholds to adapt to system changes; difficult to handle complex patterns: fixed thresholds cannot capture complex performance issues and fault patterns. Rule-based fault detection (e.g., Ji, C., Bai, W.,&Li, L. (2021). "A rule-based approach to detecting performance anomalies in cloud computing systems." IEEE Access, 9, 19260-19274) uses a predefined set of rules to detect anomalies and faults, and the rules are usually based on expert knowledge and historical data. The main problems of this approach are: as the system is updated and changed, the rules need to be constantly adjusted and maintained; it cannot cope with new abnormal patterns and unknown faults; a large amount of domain knowledge is required to define effective rules, which increases labor costs. Methods based on statistical analysis (e.g., Kim, H., Kim, S., & Kim, H. (2018). "A statistical approach to performance anomaly detection in cloud applications." Journal of Parallel and Distributed Computing, 119, 83-94) use statistical analysis methods (such as mean, variance, time series analysis, etc.) to monitor system performance and detect anomalies, establish baselines by analyzing historical data, and detect deviations from the baseline. The main problem with this method is that it requires a large amount of high-quality historical data for analysis and cannot capture rapidly changing anomalies in a timely manner. Methods based on clustering analysis (e.g., Liu, S., Zhou, M., & Li, S. (2021). "A clustering-based approach for cloud performance anomaly detection and localization." IEEE Transactions on Cloud Computing, 9(1), 210-224) use clustering algorithms (such as K-means, hierarchical clustering, DBSCAN, etc.) to discover patterns and anomalies in data. The main problems of this method include: the clustering algorithm is sensitive to parameters (such as the number of clusters, distance measurement, etc.), and improper parameter settings may lead to poor results.

[0004] Based on this, the present invention provides a new method for application performance prediction and fault detection that is more reliable and effective. Summary of the invention

[0005] The purpose of the present invention is to provide an application performance prediction and fault detection method, system and storage medium based on application clustering in view of the deficiencies of the prior art.

[0006] The technical solution adopted by the present invention is as follows: The application performance prediction and fault detection method based on application clustering includes: Collect the underlying data of the system equipment of the node to be analyzed and the application performance data and application response time data of the application layer; Cluster all collected application performance data into several categories; For all applications in each category, a prediction model for predicting application response time is built and trained based on the collected data; Afterwards, for a certain type of application, a corresponding prediction model is selected to predict the application response time of a new set of device underlying data to obtain the prediction result, and at the same time, the application response time of the application is collected; A logistic regression binary classification model is constructed based on the application response time prediction results and the corresponding collected application response time, the collected data of the corresponding type of application is manually labeled and classified into two categories: normal and faulty, and the logistic regression binary classification model is trained; The prediction model and logistic regression binary classification model for each type of application are jointly run to perform individual application performance prediction and fault detection for each application running on the system.

[0007] In the above technical solution, further, the underlying data of the system device is the time series data of the system's CPU usage, memory usage, disk IO load, and network bandwidth utilization; the application performance data of the application layer is the CPU usage, memory usage, disk IO load, and network bandwidth utilization of the process.

[0008] Furthermore, the Kmeans algorithm is used to cluster all collected application performance data, including: Step 1: collect application performance data for a certain application An, that is, the CPU usage time series vector CPU An , memory usage time series vector MEM An , Disk IO load timing vector IO An , Network bandwidth utilization time series vector NET An; Decompose and combine according to the time sequence, form a vector with the data at the same time, and then obtain the reorganized vector (CPU An1 , MEM An1 , IO An1 , NET An1 )、(CPU An2 , MEM An2 , IO An2 , NET An2 )……(CPU Ant , MEM Ant , IO Ant , NET Ant ); Among them, CPU An1 、CPU An2 ...CPU Ant for CPU An The data elements at different times in the same way; Step 2: Repeat the method in step 1 for all applications to obtain a vector set of all applications; Step 3: Run the Kmeans algorithm on all vector sets obtained in step 2 to cluster the vectors into four categories: CPU-based, memory-based, hard disk-based, and network-based. Step 4: for each application An, count the timing performance data vectors associated with it and classify them into the largest category among the four categories, which is the type of the application An.

[0009] Furthermore, the Kmeans algorithm is run in step 3 as follows: (1) For all application vector data (CPU Ant , MEM Ant , IO Ant , NET Ant ), where m is the application number and t is the time number, four vectors are randomly selected as four cluster centers, and the distances between the vector data and the four cluster centers are calculated using the distance method between vectors; (2) Each vector data is divided into the class with the closest distance, and then the average distance between all the application vector data in this class and its cluster center is calculated. The vector data corresponding to the average value is taken as the center, and the distance of all vector data to the center is recalculated, and the categories are reclassified according to the nearest center; (3) Repeat the process (2) until the classification categories no longer change and obtain the clustering results.

[0010] Furthermore, the construction of a prediction model for predicting application response time specifically includes: for all applications in each category, a logistic regression model is established with the collected underlying data of the system equipment and the application response time data, and the prediction model is trained with the application response time data as output; and a prediction model corresponding to each group of categories is obtained.

[0011] Furthermore, data is collected using zabbix or SNMP software.

[0012] An application performance prediction and fault detection system based on application clustering, comprising: A data collection module is used to collect the underlying data of the system equipment of the node to be analyzed and the application performance data and application response time data of the application layer; The clustering module is used to cluster all collected application performance data into several categories; The application response prediction module establishes a logistic regression model with the collected system device underlying data for all applications of each category, and trains the prediction model with the application response time data as output; obtains the prediction model corresponding to each group of categories; for a certain type of application, selects the corresponding prediction model to predict the application response time of a new set of device underlying data, obtains the application response time prediction result, and at the same time, collects the application response time of the application; The fault detection module is used to construct a logistic regression binary classification model for the application response time prediction result and the application response time, manually annotate the collected data of the corresponding type of application, classify it into normal and faulty categories, train the logistic regression binary classification model; and perform fault detection based on the constructed logistic regression binary classification model.

[0013] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the application performance prediction and fault detection method based on application clustering as described in any one of the above items.

[0014] An electronic device, comprising: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the application performance prediction and fault detection method based on application clustering as described in any one of the above items.

[0015] The beneficial effects of the present invention are: The present invention can effectively improve the prediction accuracy and universality by directly using application-related performance data through an application performance prediction and fault detection method based on application clustering. The application clustering method can identify different types of applications and workload patterns by clustering and analyzing historical performance data. The applications in each cluster have similar performance characteristics and behavior patterns, which makes the performance prediction of applications in a specific cluster more accurate, reduces the occurrence of false positives and false negatives, and improves the reliability and effectiveness of the prediction. Application performance prediction after clustering can improve the scalability of the model, so that the model can remain effective in a large-scale and dynamically changing cloud environment and adapt to different application scenarios and requirements. At the same time, predicting the performance of the application and performing fault prediction based on the system's total CPU, memory, IO and network indicators can reflect the operating status of a specific application in the entire system environment from the overall resource usage. By analyzing these overall indicators, the performance changes of a specific application can be predicted more accurately, so that the prediction model can better reflect the performance of a specific application under the overall system load, reduce false positives and false negatives, and improve the accuracy and reliability of the prediction. Fault detection through the overall system performance indicators can more quickly and accurately diagnose the performance problems of a specific application, shorten the troubleshooting time, and reduce the impact on the normal operation of the application. Starting with the analysis of the system's total CPU, memory, IO and network indicators, optimization suggestions can be provided for specific applications, such as adjusting resource allocation, optimizing application configuration or improving architecture design, thereby improving application performance and resource utilization efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A flowchart of clustering applications using Kmeans in the present invention; Figure 2 It is a schematic diagram of the overall method flow of the present invention; Figure 3 Schematic diagram of training and prediction of the prediction model in the present invention. DETAILED DESCRIPTION

[0017] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0018] like Figures 1 to 3 As shown, according to a specific embodiment of the present invention, an application performance prediction and fault detection method based on application clustering includes: Step 1: Use tools such as Zabbix and SNMP to collect the system CPU of each node in the network every 10 seconds. CPU S ),Memory( MEM S ),harddisk( IO S ),network( NETS ) and other device underlying data and application layer CPU ( CPU An ),Memory( MEM An ),harddisk( IO An ),network( NET An ) and application response time data ( RESP An ), it is worth noting that the italicized CPU S The indicators are all vector data, such as CPU S =(CPU S1 , CPU S2 , CPU S3 , CPU S4 , ...CPU St-1 , CPU St ); Step 2: Use the Kmeans clustering algorithm to collect all application performance data CPU An , MEM An , IO An , NET An Clustering is performed and divided into four categories: CPU-based, memory-based, hard disk-based, and network-based.

[0019] Step 3: Collect the system bottom layer data for the above four categories of applications. CPU S , MEM S , IO S , NET S Data and application response time RESP An data.

[0020] Step 4: For all applications in each of the above four categories, such as CPU-based, memory-based, hard disk-based or network-based applications, use the device underlying data ( CPU S , MEM S , IO S , NET S ) Establish a logistic regression model Mi to predict RESP AnAs a result, the prediction model is trained, that is, four prediction models M1, M2, M3, and M4 are established for CPU-based, memory-based, hard disk-based, or network-based applications respectively.

[0021] Step 5: After the prediction model Mi has been trained for a certain number of rounds, the parameters of the model Mi are fixed. According to an embodiment of the present invention, the training and prediction process of the prediction model can be as follows: Figure 3 shown.

[0022] Step 6: When the prediction model runs inference, for a certain type of application An, select the corresponding prediction model Mi for a new set of device underlying data (CPU St MEM St ,IO St , NET St ) Perform application performance prediction on An application and obtain the application response time prediction result RESP Anpt At the same time, collect the application response time RESP of the application At After a certain amount of running time, a series of results are obtained, forming a vector set CPU S , MEM S , IO S , NET S , RESP Anp , RESP An .

[0023] Step 7: For the vector set in step 6 RESP Anp , RESP An Construct a logistic regression binary classification model C.

[0024] Step 8: RESP Anp , RESP An The vector set is manually labeled and divided into two categories: normal and error.

[0025] Step 9: Train the logistic regression binary classification model C on the manually labeled data. After the logistic regression binary classification model C has undergone a certain number of training rounds, the parameters of the model C are fixed.

[0026] Step 10, jointly running the above-mentioned data collection software, the prediction model for each type of application and the logistic regression binary classification model, can perform separate application performance prediction and fault detection for each application running on the system.

[0027] Wherein, the system CPU ( CPU S ),Memory( MEM S ),harddisk( IO S ),network( NET S ) and other device bottom-level data usually refer to the total CPU usage, total memory usage, total hard disk read and write rate, total network transmission rate of the system layer, and the CPU ( CPU An ),Memory( MEM An ),harddisk( IO An ),network( NET An ) usually refers to the CPU usage, memory usage, hard disk read and write rate, network transmission rate, and application response time data of a certain application ( RESP An ) refers to the time it takes for an application to receive an access request, process all logic, and return the result.

[0028] The Kmeans clustering method specifically includes: Step 1: collect the timing performance data of a certain application An. CPU An , MEM An , IO An , NET An Decompose and combine to form a vector (CPU An1 , MEM An1 , IO An1 , NET An1 )、(CPU An2 , MEM An2 , IO An2 , NET An2 )……(CPU Ant , MEM Ant , IO Ant , NET Ant ).

[0029] Step 2: Repeat the method in step 1 for all applications to obtain the vector set of all applications (CPU A11 , MEM A11 , IO A11 , NET A11 )、(CPU A12 , MEM A12 , IOA12 , NET A12 )……(CPU A1t , MEM A1t , IO A1t , NET A1t ) and (CPU Am1 , MEM Am1 , IO Am1 , NET Am1 )、(CPU Am2 , MEM Am2 , IO Am2 , NET Am2 )……(CPU Amt , MEM Amt , IO Amt , NET Amt ) and (CPU An1 , MEM An1 , IO An1 , NET An1 )、(CPU An2 , MEM An2 , IO An2 , NET An2 )……(CPU Ant , MEM Ant , IO Ant , NET Ant ).

[0030] Step 3: Run the Kmeans algorithm on all vector sets obtained in step 2 to cluster the above vectors into four categories: CPU-based, memory-based, hard disk-based, and network-based. Figure 1 As shown. Includes: (1) For all application vector data (CPU Ant , MEM Ant , IO Ant , NET Ant ), where m is the application number and t is the time number, four vectors are randomly selected as four cluster centers, and the distances between the vector data and the four cluster centers are calculated using the distance method between vectors; (2) Each vector data is divided into the class with the closest distance, and then the average distance between all the application vector data in this class and its cluster center is calculated. The vector data corresponding to the average value is taken as the center, and the distance of all vector data to the center is recalculated, and the categories are reclassified according to the nearest center; (3) Repeat the process (2) until the classification categories no longer change and obtain the clustering results.

[0031] Step 4: for each application An, count the timing performance data vectors associated with it and classify them into the largest category among the four categories, which is the application type of application An.

[0032] The application performance prediction and fault detection method based on application clustering can better identify and utilize the similarities and patterns between applications, increase training sample data, and thus achieve more accurate and efficient performance prediction and fault detection. At the same time, due to the global perspective and detailed data provided by the overall system performance indicators, the prediction and detection methods can more comprehensively and accurately reflect the actual status and potential problems of specific applications in the overall system operating environment. Performance prediction and fault detection of specific applications based on the system's total CPU, memory, IO and network indicators can improve prediction accuracy, identify resource competition problems, discover performance bottlenecks in advance, and improve fault diagnosis efficiency. It provides an effective solution for improving application performance prediction accuracy, improving fault detection efficiency, and optimizing resource allocation in cloud computing environments.

[0033] The present invention also provides an application performance prediction and fault detection system based on application clustering, comprising: A data collection module is used to collect the underlying data of the system equipment of the node to be analyzed and the application performance data and application response time data of the application layer; The clustering module is used to cluster all collected application performance data into several categories; The application response prediction module establishes a logistic regression model with the collected system device underlying data for all applications of each category, and trains the prediction model with the application response time data as output; obtains the prediction model corresponding to each group of categories; for a certain type of application, selects the corresponding prediction model to predict the application response time of a new set of device underlying data, obtains the application response time prediction result, and at the same time, collects the application response time of the application; The fault detection module is used to construct a logistic regression binary classification model for the application response time prediction result and the application response time, manually annotate the collected data of the corresponding type of application, classify it into normal and faulty categories, train the logistic regression binary classification model; and perform fault detection based on the constructed logistic regression binary classification model.

[0034] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0035] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0036] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0037] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0038] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention are included in the protection scope of the present invention.

Claims

1. An application performance prediction and fault detection method based on application clustering, characterized in that: include: Collect the underlying data of the system equipment of the node to be analyzed and the application performance data and application response time data of the application layer; Cluster all collected application performance data into several categories; For all applications in each category, a prediction model for predicting application response time is built and trained based on the collected data; Afterwards, for a certain type of application, a corresponding prediction model is selected to predict the application response time of a new set of device underlying data to obtain the prediction result, and at the same time, the application response time of the application is collected; A logistic regression binary classification model is constructed based on the application response time prediction results and the corresponding collected application response time, the collected data of the corresponding type of application is manually labeled and classified into two categories: normal and faulty, and the logistic regression binary classification model is trained; The prediction model and logistic regression binary classification model for each type of application are jointly run to perform individual application performance prediction and fault detection for each application running on the system.

2. The application performance prediction and fault detection method based on application clustering according to claim 1 is characterized in that: The underlying data of the system device is the time series data of the system's CPU usage, memory usage, disk IO load, and network bandwidth utilization; the application performance data of the application layer is the CPU usage, memory usage, disk IO load, and network bandwidth utilization of the process.

3. The application performance prediction and fault detection method based on application clustering according to claim 2 is characterized in that: The Kmeans algorithm is used to cluster all collected application performance data, including: Step 1: collect application performance data for a certain application An, that is, the CPU usage time series vector CPU An , memory usage time series vector MEM An , Disk IO load timing vector IO An , Network bandwidth utilization time series vector NET An ; Decompose and combine according to the time sequence, form a vector with the data at the same time, and then obtain the reorganized vector (CPU An1 , MEM An1 , IO An1 , NET An1 )、(CPU An2 , MEM An2 , IO An2 , NET An2 )……(CPU Ant , MEM Ant , IO Ant , NET Ant ); Among them, CPU An1 、CPU An2 ...CPU Ant for CPU An The data elements at different times in the same way; Step 2: Repeat the method in step 1 for all applications to obtain a vector set of all applications; Step 3: Run the Kmeans algorithm on all vector sets obtained in step 2 to cluster the vectors into four categories: CPU-based, memory-based, hard disk-based, and network-based. Step 4: for each application An, count the timing performance data vectors associated with it and classify them into the largest category among the four categories, which is the type of the application An.

4. The application performance prediction and fault detection method based on application clustering according to claim 3 is characterized in that: The specific steps of running the Kmeans algorithm in step 3 are as follows: (1) For vector data of all applications (CPU Ant , MEM Ant , IO Ant , NET Ant ), where m is the application number and t is the time number, four vectors are randomly selected as four cluster centers, and the distances between the vector data and the four cluster centers are calculated using the distance method between vectors; (2) Each vector data is divided into the class with the closest distance, and then the average distance between all the application vector data in this class and its cluster center is calculated. The vector data corresponding to the average value is taken as the center, and the distance of all vector data to the center is recalculated, and the categories are reclassified according to the nearest center; (3) Repeat the process (2) until the classification categories no longer change and obtain the clustering results.

5. The application performance prediction and fault detection method based on application clustering according to claim 1, characterized in that: The construction of a prediction model for predicting application response time specifically includes: for all applications in each category, a logistic regression model is established with the collected underlying data of the system equipment and the application response time data, and the prediction model is trained with the application response time data as output; and a prediction model corresponding to each group of categories is obtained.

6. The application performance prediction and fault detection method based on application clustering according to claim 1, characterized in that: Data is collected using zabbix or SNMP software, and data is collected every 10 seconds.

7. An application performance prediction and fault detection system based on application clustering, characterized in that: include: A data collection module is used to collect the underlying data of the system equipment of the node to be analyzed and the application performance data and application response time data of the application layer; The clustering module is used to cluster all collected application performance data into several categories; The application response prediction module builds a logistic regression model based on the collected underlying data of system devices for all applications in each category, and uses the application response time data as output to train the prediction model; Obtain the prediction model corresponding to each group of categories; for a certain type of application, select the corresponding prediction model to predict the application response time of a new set of device underlying data, obtain the application response time prediction result, and at the same time, collect the application response time of the application; A fault detection module is used to construct a logistic regression binary classification model for the application response time prediction result and the application response time, manually annotate the collected data of the corresponding type of application, classify them into two categories: normal and faulty, and train the logistic regression binary classification model; Fault detection is performed based on the constructed logistic regression binary classification model.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the application performance prediction and fault detection method based on application clustering as described in any one of claims 1 to 6 is implemented.

9. An electronic device, characterized in that: The device comprises: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the application performance prediction and fault detection method based on application clustering as described in any one of claims 1-6.

Images