A method for analyzing the root causes of business application system failures based on deep learning

Abstract

The present application provides a method for analyzing the root causes of business application system failures based on deep learning, including S1, constructing an application system architecture; S2, constructing the relationship between each software and hardware resource in the application system and each level of the system; S3, collecting and processing the performance data of each software and hardware unit; S4, setting the performance indicator threshold of the application system; S5, fusing the multi-dimensional performance data of the application system; S6, performing spatiotemporal correlation on the multi-dimensional performance data; S7, analyzing the multi-dimensional power application correlation analysis data. The present invention first constructs the application system architecture, and performs data preprocessing, multi-dimensional fusion and simulated dynamic adjustment through the data acquisition and processing layer, so that the root causes of failures in the network layer, server layer, software layer and application layer can be combined and analyzed, so that the alarms between the layers are combined for analysis, so that the analysis results are more accurate and the operation and maintenance efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of power business application supervision, and in particular to a business application system fault root cause analysis method based on deep learning. Background Art

[0002] With the continuous development of IT technology, the composition of power application systems has become more and more complex, including multiple units such as computer room environment, power supply system, physical server, network equipment, cloud resource pool, storage, database, middleware, software system, etc. Each component unit of the power application system contains its complex structure. When a power application system fails, it is extremely difficult to quickly locate the cause of the failure. Various IT resource performance monitoring software has been deployed in the existing power system, which can monitor different components. However, the impact of alarms between different units on the production of the power application system cannot be eliminated, which will lead to misjudgment of the location of the root cause of the failure. Summary of the invention

[0003] The purpose of the present invention is to provide a method for analyzing the root causes of business application system failures based on deep learning to solve the problems raised in the above-mentioned background technology.

[0004] To achieve the above object, the present invention provides the following technical solution: a method for analyzing the root causes of business application system failures based on deep learning, the analysis method comprising the following steps:

[0005] S1, building an application system architecture and building relationships between the various levels of the system architecture, the system architecture includes a data collection and processing layer, a network layer, a server layer, a software layer, and an application layer;

[0006] S2, build the relationship between each software and hardware resource in the application system and each level of the system, and correspond each unit of the application system to the corresponding business level;

[0007] S3, collects and processes the performance data of each software and hardware unit through the data collection and processing layer of the application system;

[0008] S4, setting the performance indicator threshold of the application system, selecting the corresponding business level, and completing the threshold setting, including the static threshold and the dynamic threshold, and the static threshold has a higher priority than the dynamic threshold;

[0009] S5, integrates multi-dimensional performance data of the application system to generate application system data, including cleaning and filtering of performance data;

[0010] S6, performing spatiotemporal correlation on the multi-dimensional performance data, generating multi-dimensional power application correlation analysis data, and generating a business monitoring degree score according to a business analysis algorithm;

[0011] S7, analyze the multi-dimensional power application correlation analysis data and locate the root cause of the power application system failure.

[0012] Preferably, the establishment of the data acquisition and processing layer first requires the deployment of sensors and monitoring equipment in the computer room, and the collection of equipment operation data, temperature data and humidity data.

[0013] Preferably, the network layer displays the network devices connected to the application server, the server layer displays the operating system and virtual host resources, the software layer displays the database and middleware resources, and the application layer displays the application icons and names.

[0014] Preferably, the collection of data in S3 needs to consider the installation location of the equipment and the sensor type, and the data processing is to transmit the collected data to the data processing center, store it in the data warehouse or database, and perform preprocessing, cleaning and feature extraction.

[0015] Preferably, the static threshold and the dynamic threshold in S4 are based on the business management dimension, wherein the static threshold is manually set and the dynamic threshold is dynamically set and can be modified according to actual needs.

[0016] Preferably, after the static threshold is set, the generation of the alarm is determined by the manual threshold range;

[0017] The dynamic threshold is first calculated based on the historical indicator performance value and then calculated according to the algorithm. At the same time, the alarm suppression configuration is added and the suppression period is configured. When the same alarm appears within the period, it is suppressed.

[0018] Preferably, the specific operations of cleaning and filtering the performance data in S5 include the following steps:

[0019] A1, collects data, collecting performance data of the network layer, server layer, software layer and application layer from different software and hardware resources;

[0020] A2, perform data cleaning to remove duplicate data, missing data, and abnormal data;

[0021] A3, filter the data and filter out abnormal data according to the set performance indicator threshold, including data exceeding the threshold and data with abnormal fluctuations;

[0022] A4, performs data aggregation, aggregates the cleaned and filtered performance data, and generates four layers of data;

[0023] A5, performs data processing based on the application system model. According to the model and structure of the application system, the performance data is processed and integrated to generate four layers of data based on the application system model.

[0024] Preferably, the performance data of the network layer includes availability and throughput;

[0025] The performance data of the server layer includes CPU utilization, memory utilization, disk I / O times and availability;

[0026] The performance data of the software layer includes database key indicators and middleware key indicators, wherein the database key indicators include qps (number of queries processed per second), database cpu utilization, database memory utilization, IOPS (number of I / O operations performed by the disk per second) and availability, and the middleware key indicators include memory utilization, total number of connections and availability;

[0027] The performance data of the application layer includes concurrent transaction volume, new transaction volume, application response rate, response time and availability.

[0028] Preferably, the generation of multi-dimensional power application association analysis data and the generation of business monitoring degree scores in S6 include the following steps:

[0029] B1, analyze the spatiotemporal correlation of performance data of different dimensions, including using time series analysis, collaborative filtering and spatial correlation;

[0030] B2, based on the results of spatiotemporal correlation analysis, generate multi-dimensional power application correlation analysis data, including the degree of correlation and impact between data;

[0031] B3, design corresponding business analysis algorithms according to the business requirements and characteristics of the power application system;

[0032] B4, using business analysis algorithms to process multi-dimensional correlation analysis data and generate corresponding business monitoring scores;

[0033] B5, based on the results of the business monitoring score, monitor the operation of the power system in real time, and provide corresponding feedback and adjustments based on the score results.

[0034] Preferably, the fault locating method in S7 includes displaying all associated resource alarms, key indicator alarms and business application penalty alarms of the business application.

[0035] Technical effects and advantages of the present invention:

[0036] The present invention first constructs an application system architecture, and performs data preprocessing, multi-dimensional fusion and simulated dynamic adjustment through the data acquisition and processing layer, so that the root causes of faults in the network layer, server layer, software layer and application layer can be combined for analysis. In this way, the alarms between the layers are combined for analysis, making the analysis results more accurate and improving the operation and maintenance efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 The figure is a flowchart of the analysis method of the present invention.

[0038] Figure 2 The present invention is an operational flow chart for cleaning and filtering performance data.

[0039] Figure 3 This is a flow chart for generating multi-dimensional power application correlation analysis data of the present invention. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] The present invention provides Figure 1 A method for analyzing the root causes of business application system failures based on deep learning is shown, and the analysis method includes the following steps:

[0042] S1, build the application system architecture and the relationship between the various levels of the system architecture. The system architecture includes data collection and processing layer, network layer, server layer, software layer and application layer;

[0043] Specifically, the establishment of the data collection and processing layer first requires the deployment of sensors and monitoring equipment in the computer room, and the collection of equipment operation data, temperature data, and humidity data.

[0044] It should be noted that the network layer: the composition of the network structure; the server layer: the hardware support of the business system; the software layer: the main composition of the application system; the application layer: the response status of the application system.

[0045] S2, build the relationship between each software and hardware resource in the application system and each level of the system, and correspond each unit of the application system to the corresponding business level;

[0046] Specifically, the network layer displays the network devices connected to the application server, namely the access switch, the server layer displays the operating system and virtual host resources, the software layer displays the database and middleware resources, and the application layer displays the application icons and names.

[0047] S3, collects and processes the performance data of each software and hardware unit through the data collection and processing layer of the application system;

[0048] Specifically, the data collection in S3 needs to consider the installation location of the equipment and the sensor type. The data processing is to transmit the collected data to the data processing center, store it in the data warehouse or database, and perform preprocessing, cleaning and feature extraction.

[0049] S4, setting the performance indicator threshold of the application system, selecting the corresponding business level, and completing the threshold setting, including static threshold and dynamic threshold, and the static threshold has a higher priority than the dynamic threshold;

[0050] Specifically, the static threshold and the dynamic threshold in S4 are based on the business management dimension, wherein the static threshold is manually set and the dynamic threshold is dynamically set and can be modified according to actual needs.

[0051] Furthermore, after the static threshold is set, the generation of alarms is determined by the manual threshold range;

[0052] The dynamic threshold is first based on the historical indicator performance value, and then calculated according to the algorithm. At the same time, the alarm suppression configuration is added and the suppression period is configured. When the same alarm appears within the period, it is suppressed and only one alarm message is generated. After the configuration is completed, the business application view is built. If there is a relationship between assets, the relationship connection is automatically generated. If the relationship is not configured, it is necessary to manually connect the relationship and finally synchronize the relationship to the CMDB. Through such a configuration, the alarm information can be merged and processed, which also reduces the amount of data processing and improves the operation and maintenance efficiency.

[0053] like Figure 2 As shown, S5 integrates the multi-dimensional performance data of the application system to generate application system data, including cleaning and filtering of the performance data;

[0054] Specifically, the specific operations of cleaning and filtering performance data in S5 include the following steps:

[0055] A1, collects data, collecting performance data of the network layer, server layer, software layer and application layer from different software and hardware resources;

[0056] Furthermore, network layer performance data includes availability and throughput;

[0057] Performance data at the server layer includes CPU utilization, memory utilization, disk I / O times, and availability;

[0058] The performance data of the software layer includes database key indicators and middleware key indicators. The database key indicators include qps (number of queries processed per second), database cpu utilization, database memory utilization, IOPS (number of I / O operations per second on the disk) and availability. The middleware key indicators include memory utilization, total number of connections and availability.

[0059] Application layer performance data includes concurrent transaction volume, new transaction volume, application response rate, response time and availability.

[0060] A2, perform data cleaning to remove duplicate data, missing data, and abnormal data;

[0061] A3, filter the data and filter out abnormal data according to the set performance indicator threshold, including data exceeding the threshold and data with abnormal fluctuations;

[0062] A4, performs data aggregation, aggregates the cleaned and filtered performance data, and generates four layers of data;

[0063] A5, performs data processing based on the application system model. According to the model and structure of the application system, the performance data is processed and integrated to generate four layers of data based on the application system model.

[0064] like Figure 3 As shown, S6, the multi-dimensional performance data is temporally and spatially correlated to generate multi-dimensional power application correlation analysis data, and a business monitoring degree score is generated according to a business analysis algorithm;

[0065] Specifically, the generation of multi-dimensional power application correlation analysis data and business monitoring score in S6 includes the following steps:

[0066] B1, analyze the spatiotemporal correlation of performance data of different dimensions, including using time series analysis, collaborative filtering and spatial correlation;

[0067] B2, based on the results of spatiotemporal correlation analysis, generate multi-dimensional power application correlation analysis data, including the degree of correlation and impact between data;

[0068] B3, design corresponding business analysis algorithms according to the business requirements and characteristics of the power application system;

[0069] B4, using business analysis algorithms to process multi-dimensional correlation analysis data and generate corresponding business monitoring scores;

[0070] B5, based on the results of the business monitoring score, monitor the operation of the power system in real time, and provide corresponding feedback and adjustments based on the score results.

[0071] In the first embodiment, the algorithms for business applications at each layer are as follows:

[0072] Business application layer: The business health should be directly reflected in the business layer, so the deduction weight of the business layer is 100%. When the response time of a single probe reaches the warning level once, the business health will be deducted by 20 points. If it reaches the warning level continuously, the deduction will be accumulated, and 5 points will be added each time. That is, 20 points will be deducted for the first time, 25 points for the second time, 30 points for the third time, and a maximum of 40 points will be deducted. When the business is restored, the deduction will be reduced by 5 points when it is normal once, until there is no deduction. For multiple probes, the weighted average algorithm is used, and the deduction will be at least 10 points each time. If it is less than 10 points, it will be deducted as 10 points. That is, if there are 4 probes, the first alarm is calculated to be 5 points, but the actual deduction is 10 points; the second alarm should be deducted by 10 points, but the actual deduction is still 10 points. The deduction of 10 points should be calculated as 10 points when the cumulative total of all deduction items is less than 10 points; the business response of traffic analysis will be directly deducted by 40 points, and the indicators are concurrent transaction volume, new transaction volume, and application response rate. If there is a serious warning in response time and traffic analysis, the business will be deducted by 90 points. The response time of multiple probes also uses the weighted average calculation method. If an alarm occurs in the availability of the response time, 100 points will be directly deducted, indicating that the business application is unavailable.

[0073] Software layer: When a single resource in the software layer has an alarm level indicator, 10 points will be deducted, and the maximum cumulative deduction is 30 points. If a serious alarm occurs in the software layer, 60 points will be deducted directly, and no accumulation will be made.

[0074] Server layer: Alarm level, 10 points will be deducted for each indicator, with a maximum deduction of 20 points. Severe alarm, 40 points will be deducted, not cumulative.

[0075] Network layer: Alarm level, 10 points are deducted for each indicator, with a maximum of 20 points. Severe alarm, 40 points are deducted, and no accumulation. The indicator is interface traffic. According to the time when the alarm occurs in response time, check the interface traffic size. If the traffic is greater than the threshold (the threshold is the normal value of traffic). When the accumulated deductions at each level exceed 100, only the health level is displayed as 0. If the deduction reaches 0, the response time can still access +20 points.

[0076] S7, analyze the multi-dimensional power application correlation analysis data and locate the root cause of the power application system failure.

[0077] Specifically, the fault location method in S7 includes displaying all associated resource alarms, key indicator alarms and business application penalty alarms of the business application.

[0078] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for analyzing the root causes of business application system failures based on deep learning, characterized in that: The analytical method comprises the following specific steps: S1, building an application system architecture and building relationships between the various levels of the system architecture, the system architecture includes a data collection and processing layer, a network layer, a server layer, a software layer, and an application layer; S2, build the relationship between each software and hardware resource in the application system and each level of the system, and correspond each unit of the application system to the corresponding business level; S3, collects and processes the performance data of each software and hardware unit through the data collection and processing layer of the application system; S4, setting the performance indicator threshold of the application system, selecting the corresponding business level, and completing the threshold setting, including the static threshold and the dynamic threshold, and the static threshold has a higher priority than the dynamic threshold; S5, integrates multi-dimensional performance data of the application system to generate application system data, including cleaning and filtering of performance data; S6, performing spatiotemporal correlation on the multi-dimensional performance data, generating multi-dimensional power application correlation analysis data, and generating a business monitoring degree score according to a business analysis algorithm; S7, analyze the multi-dimensional power application correlation analysis data and locate the root cause of the power application system failure.

2. According to the deep learning-based business application system fault root cause analysis method of claim 1, it is characterized in that: The establishment of the data acquisition and processing layer first requires the deployment of sensors and monitoring equipment in the computer room, and the collection of equipment operation data, temperature data, and humidity data.

3. According to the deep learning-based business application system fault root cause analysis method of claim 1, it is characterized in that: The network layer displays the network devices connected to the application server, the server layer displays the operating system and virtual host resources, the software layer displays the database and middleware resources, and the application layer displays the application icons and names.

4. According to the deep learning-based business application system fault root cause analysis method of claim 1, it is characterized in that: The data collection in S3 needs to consider the installation location of the equipment and the sensor type. The data processing is to transmit the collected data to the data processing center, store it in the data warehouse or database, and perform preprocessing, cleaning and feature extraction.

5. According to the deep learning-based business application system fault root cause analysis method of claim 1, it is characterized in that: The static threshold and the dynamic threshold in S4 are based on the business management dimension, wherein the static threshold is manually set, and the dynamic threshold is dynamically set and can be modified according to actual needs.

6. A method for analyzing root causes of business application system failures based on deep learning according to claim 5, characterized in that: After the static threshold is set, the generation of the alarm is determined by the manual threshold range; The dynamic threshold is first calculated based on the historical indicator performance value and then calculated according to the algorithm. At the same time, the alarm suppression configuration is added and the suppression period is configured. When the same alarm appears within the period, it is suppressed.

7. The method for analyzing the root cause of failures in a business application system based on deep learning according to claim 1, characterized in that: The specific operation of cleaning and filtering the performance data in S5 includes the following specific steps: A1, collects data, collecting performance data of the network layer, server layer, software layer and application layer from different software and hardware resources; A2, perform data cleaning to remove duplicate data, missing data, and abnormal data; A3, filter the data and filter out abnormal data according to the set performance indicator threshold, including data exceeding the threshold and data with abnormal fluctuations; A4, performs data aggregation, aggregates the cleaned and filtered performance data, and generates four layers of data; A5, performs data processing based on the application system model. According to the model and structure of the application system, the performance data is processed and integrated to generate four layers of data based on the application system model.

8. The method for analyzing the root causes of failures in a business application system based on deep learning according to claim 7, characterized in that: The performance data of the network layer includes availability and throughput; The performance data of the server layer includes CPU utilization, memory utilization, disk I / O times and availability; The performance data of the software layer includes database key indicators and middleware key indicators, wherein the database key indicators include qps (number of queries processed per second), database cpu utilization, database memory utilization, IOPS (number of I / O operations performed by the disk per second) and availability, and the middleware key indicators include memory utilization, total number of connections and availability; The performance data of the application layer includes concurrent transaction volume, new transaction volume, application response rate, response time and availability.

9. The method for analyzing the root cause of failures in a business application system based on deep learning according to claim 1, characterized in that: The generation of multi-dimensional power application correlation analysis data and the generation of business monitoring scores in S6 include the following specific steps: B1, analyze the spatiotemporal correlation of performance data of different dimensions, including using time series analysis, collaborative filtering and spatial correlation; B2, based on the results of spatiotemporal correlation analysis, generate multi-dimensional power application correlation analysis data, including the degree of correlation and impact between data; B3, design corresponding business analysis algorithms according to the business requirements and characteristics of the power application system; B4, using business analysis algorithms to process multi-dimensional correlation analysis data and generate corresponding business monitoring scores; B5, based on the results of the business monitoring score, monitor the operation of the power system in real time, and provide corresponding feedback and adjustments based on the score results.

10. The method for analyzing root causes of business application system failures based on deep learning according to claim 1, characterized in that: The fault location method in S7 includes displaying all associated resource alarms, key indicator alarms and business application penalty alarms of the business application.

Images