Server fault monitoring method and system based on data analysis

Abstract

The invention belongs to the technical field of data processing, and particularly relates to a server fault monitoring method and system based on data analysis, and the method comprises the steps: obtaining a performance index of a to-be-monitored server, and forming performance index time series data; extracting a short-term fluctuation index used for representing the recent operation stability of the server and a middle-term trend index used for representing the middle-term performance evolution direction of the server; combining the performance index, the short-term fluctuation index and the medium-term tendency index of the current time point, and constructing an enhanced feature vector; and inputting the enhanced feature vector into a preset anomaly detection model to generate an anomaly score for quantifying the state anomaly degree of the current server, thereby realizing fault monitoring of the server. According to the method, the time context features are constructed and utilized, so that the server fault monitoring accuracy and the early warning capability are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology. More specifically, this invention relates to a server fault monitoring method and system based on data analysis. Background Technology

[0002] For IDC and cloud computing service providers, ensuring the continuous, stable, and efficient operation of their tens of thousands of server clusters is a core competitive advantage. Server failures, especially progressive failures that go from slight performance degradation to complete downtime, have become one of the most challenging and difficult problems in data center operations due to their insidious occurrence and slow evolution.

[0003] Traditional server monitoring methods mainly rely on setting fixed alarm thresholds for key performance indicators such as CPU utilization and memory usage. However, business has a tidal effect, and fixed thresholds are difficult to adapt to normal performance fluctuations under different business loads, which can easily generate a large number of false alarms during peak business periods. Secondly, for progressive faults, such as minor memory leaks, the changes in indicators are slow and will not reach high alarm thresholds for a long time, resulting in missed reports and missing the best time for intervention.

[0004] Currently, existing technologies commonly employ machine learning-based anomaly detection algorithms, such as the Isolation Forest algorithm, which is widely used in server anomaly detection due to its efficiency in processing high-dimensional data. However, the Isolation Forest algorithm analyzes server performance data at each monitoring point in time as an isolated, static snapshot, completely ignoring the inherent correlation and evolutionary trends of data over time. In fact, the core characteristic of progressive server failures lies in the anomaly of their temporal evolution pattern, rather than a numerical anomaly at a single moment. For example, a slow memory leak will exhibit an abnormal, continuously rising trend in memory usage. Traditional Isolation Forest algorithms cannot directly model such temporal patterns because their evaluation of each data point is isolated, lacking awareness of historical and future context. This makes it difficult to detect potential failures in their early stages, posing a serious risk of service interruption to data centers. Summary of the Invention

[0005] To address the technical problem in existing technologies that make it difficult to effectively detect potential slow-onset server failures due to the neglect of time-series dependencies in performance data, this invention provides solutions in the following aspects.

[0006] In a first aspect, the present invention provides a server fault monitoring method based on data analysis, comprising: acquiring performance indicators of a server to be monitored to form time series data of performance indicators; analyzing the time series data of performance indicators based on a preset first time window to extract a short-term volatility indicator for characterizing the recent operational stability of the server; analyzing the time series data of performance indicators based on a preset second time window to extract a medium-term trend indicator for characterizing the medium-term performance evolution direction of the server; combining the performance indicators, the short-term volatility indicator, and the medium-term trend indicator at the current time point to construct an enhanced feature vector; inputting the enhanced feature vector into a preset anomaly detection model to generate an anomaly score that quantifies the degree of anomaly in the current server state, and determining whether the server is in a healthy state based on the anomaly score, thereby realizing fault monitoring of the server.

[0007] This invention first extracts temporal context features that characterize the dynamic evolution pattern from the original performance index time series data, combines these temporal features with the original index into an enhanced feature vector, and finally performs anomaly detection on the enhanced feature vector, thereby enabling early and accurate monitoring of server failures based on more comprehensive information.

[0008] Preferably, the short-term volatility index satisfies the following relationship: Where t is the current time point, This is a short-term volatility indicator at the current point in time, where N is the size of the preset first time window. The standard deviation of the performance index data within the first time window. This is the arithmetic mean of the performance metrics data within the first time window. This is a preset hyperparameter used to prevent the denominator from being zero.

[0009] This short-term volatility index can effectively quantify the recent instability of performance indicators. Compared with using only the standard deviation, it eliminates the influence of the mean of the data on the judgment of volatility and can more accurately capture the precursors of the system starting to become unstable.

[0010] Preferably, the intermediate trend index is obtained by performing linear regression analysis on the data points within the second time window, and the intermediate trend index satisfies the following relationship: Where t is the current time point, This represents the medium-term trend indicator at the current point in time, where M is the size of the preset second time window. For the index of time points within the second time window, For time points The corresponding performance index values, The average value of the time point index. This represents the average value of the performance index.

[0011] This mid-term trend indicator can intuitively and quantitatively determine the direction and rate of performance indicator evolution over a mid-term time scale, which is crucial for identifying whether a server has a tendency to continuously deteriorate.

[0012] Preferably, the process of constructing the enhanced feature vector involves horizontally concatenating the performance index, the short-term volatility index, and the medium-term trend index along the feature dimension to form a single enhanced feature vector with a dimension higher than the original performance index.

[0013] Preferably, after acquiring the time series data of the performance indicators, the method further includes performing Z-Score standardization on the time series data of the performance indicators to eliminate the dimensional differences between different performance indicators.

[0014] Standardization allows performance metrics with different physical meanings and numerical ranges to be analyzed and modeled on a uniform scale, avoiding bias caused by differences in feature scales and ensuring the fairness and accuracy of subsequent feature extraction and anomaly scoring.

[0015] Preferably, the anomaly detection model is an isolated forest model.

[0016] Preferably, determining whether the server is in a healthy state based on the abnormal score includes: assigning the server to a homogeneous server cluster according to business type or hardware configuration; calculating a dynamic alarm threshold based on the distribution of abnormal scores of all servers in the homogeneous server cluster during historical normal operation periods; and determining that the server is in an unhealthy state when the real-time abnormal score of the server is continuously higher than the dynamic alarm threshold.

[0017] By adopting dynamic alarm thresholds based on homogeneous server clusters, the criteria for judging anomalies can be adaptively adjusted according to the load characteristics of different business clusters and different time periods, thus solving the problem of monitoring baseline drift caused by business diversity and periodicity.

[0018] Preferably, the calculation of the dynamic alarm threshold includes: obtaining the statistical distribution of all abnormal scores during the historical normal operation period; and selecting a preset percentile of the statistical distribution as the dynamic alarm threshold.

[0019] Secondly, the present invention provides a server fault monitoring system based on data analysis, including a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the aforementioned server fault monitoring method based on data analysis is implemented.

[0020] By adopting the above technical solution, a computer program is generated from the above-mentioned server fault monitoring method based on data analysis and stored in the memory so that it can be loaded and executed by the processor. In this way, a terminal device can be made based on the memory and the processor for convenient use.

[0021] This invention constructs and integrates two temporal context features: short-term volatility and medium-term trend, overcoming the limitation of existing technologies that can only analyze static data snapshots. By modeling the dynamic evolution pattern of server performance, it can accurately identify progressive faults that are difficult to detect by traditional methods, such as slow resource leakage, significantly advancing the timing of fault detection and greatly improving the accuracy and foresight of fault monitoring.

[0022] Furthermore, by adopting dynamic alarm thresholds and alarm persistence judgment logic based on homogeneous clusters, the monitoring system can automatically adapt to the load characteristics of different services, thereby effectively filtering out instantaneous interference, significantly reducing the false alarm rate, ensuring the high signal-to-noise ratio and operability of alarm signals, and enhancing the reliability and robustness of server fault detection. Attached Figure Description

[0023] The above and other objects, features, and advantages of exemplary embodiments of the present invention will become readily apparent upon reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of the invention are illustrated by way of example and not limitation, and like or corresponding reference numerals denote like or corresponding parts, wherein: Figure 1 This is a flowchart illustrating a server fault monitoring method based on data analysis according to the present invention; Figure 2 This is a schematic diagram illustrating a comparison of raw data on server memory usage. Figure 3 This is a schematic diagram illustrating the monitoring effect of existing technologies; Figure 4 This is a schematic diagram illustrating the monitoring effect of the method of the present invention. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0026] This invention discloses a server fault monitoring method based on data analysis, referring to... Figure 1 This includes steps S1-S5: S1. Obtain the performance metrics of the server to be monitored and generate time series data of the performance metrics.

[0027] In an optional embodiment, a monitoring agent deployed on thousands of servers in a data center can be used to collect multi-dimensional performance metrics of each server in parallel at a sampling frequency of once per minute. These performance metrics can comprehensively reflect key parameters of server health status, such as CPU utilization (%), memory utilization (%), disk I / O latency (ms), and network bytes received per second (bytes / s).

[0028] In this optional embodiment, since these indicators have different dimensions and numerical ranges, it is necessary to standardize each piece of raw time series data collected. This embodiment preferably uses the Z-Score standardization method, which involves subtracting the mean of the sequence containing each data point and then dividing by its standard deviation. After processing, all performance indicators are transformed into a standard normal distribution with a mean of 0 and a standard deviation of 1, thereby eliminating dimensional differences and obtaining time series data of performance indicators with multiple dimensions.

[0029] like Figure 2 The diagram illustrates a comparison of raw data on server memory usage. The black dashed line represents a healthy server, whose memory usage fluctuates smoothly and randomly around a baseline of 40%. The black solid line represents a sub-healthy server, which shows a small but continuous linear increase in memory usage from 40%, indicating a slow memory leak process.

[0030] In this way, by collecting multi-dimensional performance indicators at high frequency and standardizing them with Z-Score, we can provide high-quality data support without the influence of dimensions for subsequent analysis.

[0031] S2. Based on the preset first time window, analyze the time series data of performance indicators and extract short-term volatility indicators to characterize the recent operational stability of the server.

[0032] In an optional embodiment, to assess the recent operational stability of the quantification server, a relatively short time window can be pre-set as the first time window, for example, 10 minutes, i.e., 10 time points. For the current time point t, the system dynamically extracts the performance indicator time series data from the 10 minutes preceding that time point, and extracts a short-term volatility indicator to characterize the recent operational stability of the server based on the performance indicator time series data.

[0033] Specifically, the short-term volatility indicator satisfies the following relationship:

[0034] Where t is the current time point, This is a short-term volatility indicator at the current point in time, where N is the size of the preset first time window. The standard deviation of the performance index data within the first time window. This is the arithmetic mean of the performance metrics data within the first time window. For example, a preset hyperparameter used to prevent the denominator from being zero can be provided as a reference. .

[0035] Furthermore, the short-term volatility index represents the statistical coefficient of variation. The larger the value, the more unstable the server's operating status and the more severe the fluctuations within that short time window.

[0036] For example, suppose the standardized data sequence of a server's memory usage over a 10-minute period (t-9 to t) is [-0.5, -0.3, -0.6, -0.2, 0.1, 0.0, 0.2, 0.4, 0.3, 0.5]. Then the mean of this sequence is... -0.01, standard deviation If it is 0.44, then the short-term volatility index The absolute value of the short-term volatility indicator reflects the stability of recent performance.

[0037] In this way, by constructing a short-term volatility index, key information about the recent stability of the server can be quantified from time-series data, providing early signals for the detection of impending performance problems.

[0038] S3. Based on the preset second time window, analyze the time series data of performance indicators and extract the mid-term trend index to characterize the mid-term performance evolution direction of the server.

[0039] In an optional embodiment, the medium- to long-term evolution direction of performance metrics can be captured to identify progressive failures. Therefore, a second time window can be pre-set to analyze the time series data of performance metrics and extract a medium-term trend index to characterize the medium-term performance evolution direction of the server.

[0040] Specifically, a longer second time window can be set, such as 60 minutes, or 60 time points. For the current time point t, the system extracts the time series data of the performance indicators from the previous 60 minutes and performs least squares linear regression analysis on these data points to fit a straight line that best represents its trend. The slope of this straight line is defined as the medium-term trend index.

[0041] In this optional embodiment, the medium-term trend index satisfies the following relationship:

[0042] Where t is the current time point. This represents the medium-term trend indicator at the current point in time, where M is the size of the preset second time window. For the index of time points within the second time window, For time points The corresponding performance index values, The average value of the time point index. This represents the average value of the performance index.

[0043] For example, suppose a server's memory usage data points show a clear linear growth trend over the past 60 minutes, with time indices from 1 to 60. The slope can be calculated using the relationship of the medium-term trend index. A slope of 0.08, which is clearly positive, accurately reflects the continuous deterioration trend of the performance indicator increasing by an average of 0.08 standard deviations per minute, and is a key reference for discovering potential faults such as memory leaks.

[0044] In this way, by constructing a mid-term trend indicator, it is possible to accurately capture whether there is a tendency for server performance to deteriorate continuously, and effectively identify slow, progressive failures that cannot be detected by traditional threshold methods.

[0045] S4. Combine the performance indicators, short-term volatility indicators, and medium-term trend indicators at the current time point to construct an enhanced feature vector.

[0046] In an optional embodiment, the contextual information represented by the obtained performance metrics, short-term volatility metrics, and medium-term trend metrics can be effectively fused. For the current time point t, assuming there are k original performance metrics, the system horizontally concatenates the current values ​​of these k performance metrics, the k short-term volatility metrics, and the k medium-term trend metrics along the feature dimension to form a single 3k-dimensional enhanced feature vector, the structure of which can be represented as:

[0047] in, This represents a vector concatenation operation. This enhanced feature vector not only contains the current state of the server, but also implies its recent stability and medium-term evolution trend.

[0048] Thus, by constructing an enhanced feature vector that incorporates temporal context, the subsequent anomaly detection model can make comprehensive judgments based on information from the current, recent, and long-term trends, laying a solid foundation for accurate fault monitoring.

[0049] S5. Input the enhanced feature vector into the preset anomaly detection model to generate an anomaly score that quantifies the degree of anomaly in the current server status, and determine whether the server is in a healthy state based on the anomaly score, thereby realizing the monitoring of server faults.

[0050] In an alternative embodiment, the generated enhanced feature vector can be input in real time into a pre-trained anomaly detection model, preferably an isolated forest, which outputs an anomaly score ranging from 0 to 1. The higher the score, the more the current server's state deviates from the normal mode of its cluster.

[0051] Furthermore, to determine the health status of the servers, a dynamic threshold and continuous judgment logic can be used. The system divides the servers into homogeneous server clusters and calculates a dynamic alarm threshold based on the distribution of abnormal scores during the historical normal operation of the cluster, such as the 95th percentile. Finally, the system monitors the real-time abnormal scores of the servers. Only when the score is continuously higher than the dynamic threshold for a preset duration (e.g., 5 minutes) is the server ultimately determined to be in an unhealthy state and a fault alarm is triggered; otherwise, the server is determined to be in a healthy state.

[0052] like Figure 3 and Figure 4 The diagram illustrates the monitoring effects of existing technologies and the method of the present invention. It can be seen that existing technologies cannot generate effective alarms, while... Figure 4 In the process, the anomaly score showed a continuous upward trend as the fault evolved, and eventually stably broke through the alarm threshold, achieving accurate and reliable early fault monitoring.

[0053] It is worth noting that, Figure 3 The paper demonstrates the anomaly scores generated by directly applying the standard Isolation Forest algorithm to each independent data point of a sub-healthy server. It shows that the anomaly scores of the existing technology frequently fluctuate around the threshold, failing to consistently exceed it, thus making it difficult to generate effective alerts and easily missing such slowly evolving faults. Figure 4 As time goes on, the anomaly score shows a stable and continuous upward trend. Under this multi-dimensional and strict constraint, the anomaly score calculated by the health server will be very stable and clustered in a very low value range. Therefore, the alarm threshold can be set at a lower level, thereby obtaining higher detection sensitivity without sacrificing accuracy.

[0054] Thus, by introducing an anomaly detection model and combining it with dynamic and continuous judgment logic, it is possible to achieve accurate, adaptive, and highly reliable monitoring of the server's health status.

[0055] This invention also discloses a server fault monitoring system based on data analysis, including a processor and a memory. The memory stores computer program instructions, which, when executed by the processor, implement a server fault monitoring method based on data analysis according to the present invention.

[0056] The system also includes other components well known to those skilled in the art, such as communication buses and communication interfaces, the settings and functions of which are known in the art and will not be described in detail here.

[0057] In the description of this specification, "multiple" or "several" means at least two, such as two, three or more, unless otherwise expressly and specifically defined.

[0058] While this specification has shown and described numerous embodiments of the invention, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will occur to those skilled in the art without departing from the spirit and essence of the invention. It should be understood that various alternatives to the embodiments of the invention described herein may be employed in the practice of this invention.

Claims

1. A server fault monitoring method based on data analysis, characterized in that, include: Obtain the performance metrics of the server to be monitored and generate time-series data of the performance metrics. Based on a preset first time window, the time series data of the performance indicators are analyzed to extract short-term volatility indicators that characterize the recent operational stability of the server. Based on a preset second time window, the time series data of the performance indicators are analyzed to extract a mid-term trend index that characterizes the mid-term performance evolution direction of the server. The performance indicators, short-term volatility indicators, and medium-term trend indicators at the current time point are combined to construct an enhanced feature vector; The enhanced feature vector is input into a preset anomaly detection model to generate an anomaly score that quantifies the degree of anomaly in the current server state. Based on the anomaly score, it is determined whether the server is in a healthy state, thereby realizing fault monitoring of the server.

2. The server fault monitoring method based on data analysis according to claim 1, characterized in that, The short-term volatility indicator satisfies the following relationship: Where t is the current time point, This is a short-term volatility indicator at the current point in time, where N is the size of the preset first time window. The standard deviation of the performance index data within the first time window. This is the arithmetic mean of the performance metrics data within the first time window. This is a preset hyperparameter used to prevent the denominator from being zero.

3. The server fault monitoring method based on data analysis according to claim 1, characterized in that, The intermediate-term trend index is obtained by performing linear regression analysis on data points within the second time window, and the intermediate-term trend index satisfies the following relationship: Where t is the current time point. This represents the medium-term trend indicator at the current point in time, where M is the size of the preset second time window. For the index of time points within the second time window, For time points The corresponding performance index values, The average value of the time point index. This represents the average value of the performance index.

4. The server fault monitoring method based on data analysis according to claim 1, characterized in that, The process of constructing the enhanced feature vector involves horizontally concatenating the performance index, the short-term volatility index, and the medium-term trend index along the feature dimension to form a single enhanced feature vector with a dimension higher than the original performance index.

5. The server fault monitoring method based on data analysis according to claim 1, characterized in that, After obtaining the time series data of the performance indicators, the method further includes performing Z-Score standardization on the time series data of the performance indicators to eliminate the dimensional differences between different performance indicators.

6. The server fault monitoring method based on data analysis according to claim 1, characterized in that, The anomaly detection model is the isolated forest model.

7. The server fault monitoring method based on data analysis according to claim 1, characterized in that, The step of determining whether the server is in a healthy state based on the abnormal score includes: The servers are grouped into a homogeneous server cluster according to their business type or hardware configuration. Based on the distribution of abnormal scores of all servers in the homogeneous server cluster during historical normal operation periods, the dynamic alarm threshold is calculated. When the server's real-time anomaly score remains higher than the dynamic alarm threshold, the server is determined to be in an unhealthy state.

8. The server fault monitoring method based on data analysis according to claim 7, characterized in that, The calculation of the dynamic alarm threshold includes: Obtain the statistical distribution of all abnormal scores during the historical normal operation period; A preset percentile of the statistical distribution is selected as the dynamic alarm threshold.

9. A server fault monitoring system based on data analysis, characterized in that, include: A processor and a memory, the memory storing computer program instructions that, when executed by the processor, implement a server fault monitoring method based on data analysis according to any one of claims 1-8.

Images