Server fault diagnosis method and system for server group

Abstract

The present invention relates to the field of data processing, and in particular to a server fault diagnosis method and system for a server group, the method comprising: obtaining and preprocessing server operating status data; constructing an operating status isolation tree using an isolation forest algorithm, analyzing the over-limit and under-limit conditions of data in each dimension, determining a neighbor monitoring sequence and constructing a threshold offset index; analyzing the correlation characteristics between data in each dimension based on the index to construct a fault correlation index; further analyzing the data anomaly differences between leaf nodes and child nodes in the isolation tree to construct an intra-layer isolation index; calculating anomaly isolation weights and significant anomaly weights based on the above, improving the isolation forest algorithm, and achieving accurate diagnosis of server faults. The present invention improves the isolation forest algorithm by introducing significant anomaly weights, constructs a threshold offset index and a fault correlation index, significantly improves the accuracy of server fault diagnosis, reduces false positives and negatives, helps operation and maintenance personnel accurately locate faults, and ensures stable server operation.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and more particularly to a server fault diagnosis method and system for a server group. Background Art

[0002] A server cluster is a complex system composed of numerous servers. During daily operation, each server generates massive amounts of performance and hardware monitoring data, covering key metrics such as CPU usage, memory utilization, and server temperature. Due to the high frequency of data collection and the explosive growth in data volume, this data management and analysis within the server cluster presents significant challenges.

[0003] Accurate fault diagnosis is crucial in the operation and maintenance of server clusters. Server clusters handle critical business processes, and a failure can lead to serious consequences such as service interruption and data loss. By analyzing and monitoring massive amounts of cluster data, operators can promptly detect fault signs and take proactive measures to mitigate the impact, maintain cluster stability, and ensure business continuity.

[0004] Isolation forest, a commonly used anomaly detection algorithm, is highly efficient when processing large datasets and does not require complex modeling of the entire dataset. However, this algorithm also has some shortcomings. When calculating the anomaly score, the traditional isolation forest algorithm treats the anomaly weight of each isolation tree as equal. However, in reality, when constructing an isolation tree, parameters such as the selected data samples, the split dimension of each layer, and the split threshold are all randomly selected. This results in different isolation trees having different effects in isolating anomalous data. As a result, the final calculated anomaly score may not accurately reflect the true level of anomalies in the server group, resulting in a decrease in the accuracy of server fault diagnosis. Summary of the Invention

[0005] To address the problem that the traditional isolation forest algorithm is highly efficient when processing large-scale data sets, but ignores the differences in isolation effects caused by the randomness of each isolated tree, making anomaly scores easily inaccurate and reducing the accuracy of server fault diagnosis, the present invention provides solutions in the following aspects.

[0006] In a first aspect, a server fault diagnosis method for a server group includes: obtaining and preprocessing the operating status data of the server during operation; using the isolation forest algorithm to construct an operating status isolation tree of the server, determining a neighbor monitoring over-limit sequence and a neighbor monitoring under-limit sequence according to the over-limit and under-limit conditions in the operating status data of each dimension of the server, analyzing the abnormality degree of the neighbor monitoring over-limit sequence and the neighbor monitoring under-limit sequence for constructing a threshold offset index; analyzing the correlation characteristics between the data of each dimension during the operation of the server based on the threshold offset index, constructing a fault correlation index for measuring the degree of fault correlation between data of different dimensions, analyzing the difference in the degree of data abnormality between leaf nodes and child nodes in the isolation tree based on the fault correlation index, and constructing an intra-layer isolation index; calculating the abnormal isolation weight of the isolation tree based on the intra-layer isolation index, taking the ratio of the abnormal isolation weight of each isolation tree to the sum of the abnormal isolation weights of all isolated trees in the isolation forest as the significant abnormality weight, and improving the isolation forest algorithm based on the significant abnormality weight to diagnose the server fault.

[0007] By constructing a threshold offset index and a fault correlation index, the degree of anomaly in each dimension of server data and the fault correlation between different dimensions are precisely quantified, making fault diagnosis more accurate. At the same time, anomaly isolation weights are calculated based on the intra-layer isolation index, and the isolation forest algorithm is improved to assign reasonable weights to different isolation trees, further improving the accuracy of anomaly scores, effectively avoiding misjudgments and missed detections, and providing a reliable basis for server fault diagnosis. A comprehensive analysis of server operating status data in each dimension, including system status and hardware status, is performed to comprehensively monitor server operation. Based on the analysis of the degree of anomaly in each dimension of data, the correlation characteristics between dimensions are deeply explored, and a fault correlation index is constructed to more comprehensively reflect the server's operating status. This method helps to promptly identify potential fault hazards, reduce false positives and missed detections, and improve the reliability and comprehensiveness of fault diagnosis.

[0008] Preferably, the over-limit and under-limit include:

[0009] In server operation status monitoring, over-limit refers to the server operation data in each dimension exceeding the preset upper limit, and under-limit refers to the server operation data in each dimension falling below the preset lower limit.

[0010] Preferably, determining the neighbor monitoring over-limit sequence and the neighbor monitoring under-limit sequence includes:

[0011] Taking any moment as the target moment, the data of each dimension parameter corresponding to the target moment at a preset number of moments before the target moment is used as the neighbor monitoring sequence; obtaining the monitoring data when the server is running smoothly to determine the normal range of each dimension data when running smoothly;

[0012] The data of each dimension in the nearest neighbor monitoring sequence is compared with the maximum and minimum values in the normal interval range, and the difference between the data of each dimension and the maximum value constitutes the nearest neighbor monitoring over-limit sequence, in which the data less than 0 is set to 0; the difference between the data of each dimension and the minimum value constitutes the nearest neighbor monitoring lower limit sequence, in which the data greater than 0 is set to 0.

[0013] By using the data of each dimension parameter at the target moment several times before the preset time as the neighbor monitoring sequence and comparing it with the normal range, it is possible to quickly discover whether the data of each dimension exceeds the normal range, capture potential fault signs in time, provide real-time monitoring data for subsequent fault diagnosis, and realize dynamic tracking of the server operation status; compare the data of each dimension in the neighbor monitoring sequence with the maximum and minimum values of the normal range, and set the data less than 0 or greater than 0 to 0. This processing process can effectively filter out slight deviations caused by data fluctuations, avoid misjudging normal short-term fluctuations as faults, thereby reducing the false alarm rate and improving the accuracy of fault warning.

[0014] Preferably, the threshold deviation index includes:

[0015] The length of continuous appearance of non-zero data in the neighbor monitoring over-limit sequence and the neighbor monitoring under-limit sequence is recorded as the continuous length, and the interval where the non-zero data is located is recorded as the continuous interval;

[0016] Taking any moment as the target moment, traverse and calculate the product of the continuous length corresponding to the continuous interval in all neighboring monitoring out-of-limit sequences of each dimension data at the target moment and the sum of all data in the corresponding continuous interval, and sum the product results of all intervals as the out-of-limit anomaly cumulative index;

[0017] Traverse and calculate the product of the continuous length corresponding to the continuous interval in all the neighbor monitoring lower limit sequences of each dimension of the target time and the sum of all the data in the corresponding continuous interval, and sum the product results of all intervals as the lower limit anomaly cumulative index;

[0018] The over-limit anomaly cumulative index and the lower-limit anomaly cumulative index are added together as the threshold deviation index of each dimension data at the target moment.

[0019] By calculating the continuous length and continuous interval of non-zero data in the neighbor monitoring over-limit and under-limit sequences, and combining these data to construct the over-limit anomaly cumulative index and the under-limit anomaly cumulative index, we focus not only on whether the data exceeds or under-limits, but also on the duration and cumulative impact of the abnormal state, thereby providing richer quantitative information on anomalies. The threshold offset index is obtained by adding the over-limit anomaly cumulative index and the under-limit anomaly cumulative index, which can comprehensively reflect the degree of abnormal deviation of data in each dimension at the target time. This helps to more accurately identify and diagnose server fault conditions, avoid misjudgments due to the randomness of a single indicator, and improve the accuracy of fault diagnosis. It can effectively capture the persistence and cumulative effects of abnormal conditions, help to promptly detect potential problems in the early stages of failure, and achieve refined monitoring of server operating status.

[0020] Preferably, the method for calculating the fault correlation index includes:

[0021] Get the number of dimensions of the server's running status, take any dimension data as the target data, and sequentially traverse and calculate the sum of the threshold offset indices between the target data and the other dimension data except the target data, as well as the absolute difference between the threshold offset indices between the target data and the other dimension data except the target data, and add 1 to the absolute difference as the adjustment factor;

[0022] The sum of the ratios of the sum of the threshold deviation indices and the adjustment factor is taken as the fault correlation index at each moment during the server operation.

[0023] By calculating the sum and absolute difference of the threshold offset indices between the target data and other dimensional data except the target data, the correlation and difference between the target dimension and other dimensions are comprehensively considered, thereby more comprehensively evaluating the relationship between data of different dimensions, which helps to discover potential fault modes; the sum of the ratio between the sum of the threshold offset indices and the adjustment factor is used as the fault correlation index, which can comprehensively reflect the degree of fault correlation between the data of each dimension at each moment during the server operation process.

[0024] Preferably, the intra-layer isolation index includes:

[0025] Taking any layer in the isolation tree as the layer data to be analyzed, calculate the absolute difference between the mean of the fault correlation index of the server running status data in the leaf nodes in the layer data to be analyzed and the mean of the fault correlation index of the server running status data in the non-leaf nodes in the layer data to be analyzed, and use the absolute difference as the mean difference between the nodes;

[0026] Calculate the absolute difference between the original segmentation threshold and the Otsu threshold of the server operation status of the layer data to be analyzed in the feature segmentation dimension, and add 1 to the absolute difference as the threshold adjustment factor;

[0027] The ratio between the mean difference between nodes and the threshold adjustment factor is used as the intra-layer isolation index of the layer data to be analyzed in the isolation tree.

[0028] By calculating the absolute difference between the mean values of the fault correlation index of the server operating status data within leaf nodes and non-leaf nodes in the data layer to be analyzed, the degree of anomaly in the data layer can be accurately located, helping to quickly identify nodes that may have faults in the isolation tree. By calculating the absolute difference between the original segmentation threshold and the Otsu threshold for the server operating status of the data layer to be analyzed on the feature segmentation dimension, and adding 1 to this difference as the threshold adjustment factor, the segmentation threshold can be adjusted more precisely, improving the segmentation effect of the isolation tree and helping to better distinguish normal and abnormal data when constructing the isolation tree.

[0029] Preferably, the abnormal isolation weight includes:

[0030] Taking any layer in the isolation tree as the layer data to be analyzed, the ratio between the isolation index of the layer above the layer data to be analyzed and the isolation index of the layer data to be analyzed is summed to obtain the abnormal isolation weight of the isolation book.

[0031] Preferably, the improved isolation forest algorithm based on significant anomaly weights is used to diagnose server failures, including:

[0032] The server's real-time operating status data is used as the input of the isolation forest algorithm. The significant isolation weight is used as the weight of the anomaly score of each isolated tree. The anomaly scores of all isolated trees in the isolation forest are weighted and summed to obtain a comprehensive anomaly score.

[0033] If the comprehensive anomaly score is greater than the fault threshold, it indicates that the server has a fault and should be repaired in time. Otherwise, if it is less than or equal to the fault threshold, it indicates that the server is normal.

[0034] Preferably, preprocessing the operating status data includes:

[0035] The operating status data includes system status data and hardware status data. The system status data includes: CPU occupancy rate, memory occupancy rate, network utilization rate and disk I / O utilization rate, which is used to reflect the software operating status of the server; the hardware status data includes: server temperature, fan speed, server power and CPU temperature, which is used to monitor the hardware health of the server;

[0036] The regression filling method is used to fill the missing values in the operating status data, and the standard deviation normalization method is used to normalize the filled operating status data to achieve data standardization and complete the preprocessing of the operating status data.

[0037] In a second aspect, a server fault diagnosis system for a server group includes: a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the above-mentioned server fault diagnosis method for the server group is implemented.

[0038] The present invention has the following effects:

[0039] 1. The present invention improves the isolation forest algorithm by introducing significant anomaly weights. The abnormal isolation weight of each isolated tree is calculated according to the intra-layer isolation index, and the ratio of the abnormal isolation weight to the sum of the abnormal isolation weights of all isolated trees is used as the significant anomaly weight. In this way, the actual isolation effect differences of each isolated tree can be comprehensively considered, and different isolated trees are given more representative weights, so that more accurate and reliable anomaly scores can be obtained when diagnosing server faults, effectively avoiding misjudgments and missed judgments caused by ignoring the randomness of isolated trees, and significantly improving the accuracy of server fault diagnosis.

[0040] 2. The present invention constructs a threshold offset index and a fault correlation index by conducting an in-depth analysis of the operating status data. This can not only quantify the abnormality of the server data from multiple dimensions, but also reveal the fault correlation relationship between data of different dimensions. This helps to more comprehensively reflect the operating status of the server, timely discover potential fault hazards, and avoid missed fault reports due to the neglect of abnormalities in a single dimension. Fault diagnosis based on the comprehensive analysis results can improve the reliability of the diagnostic results, reduce false alarms, and enable operation and maintenance personnel to more accurately locate the cause of the fault, take effective solutions, and ensure the stable operation of the server. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is a method flow chart of steps S1 to S4 in a server fault diagnosis method for a server group according to an embodiment of the present invention.

[0042] Figure 2 This is a structural block diagram of a server fault diagnosis system for a server group according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, but not all of the embodiments.

[0044] Reference Figure 1 A server fault diagnosis method for a server group includes steps S1 to S4, specifically as follows:

[0045] S1: Obtain the running status data of the server during operation and perform preprocessing.

[0046] During server group monitoring, the management client collects real-time server operational status data at a one-second interval, adjustable based on actual needs. This operational status data includes both system and hardware status data. System status data, including but not limited to CPU utilization, memory utilization, network utilization, and disk I / O utilization, reflects the server's software operational status. Hardware status data, including but not limited to server temperature, fan speed, server power, and CPU temperature, monitors the health of the server's hardware.

[0047] Due to factors such as environmental interference or network congestion, the collected data may contain missing values. To ensure data integrity and accuracy and avoid impacting subsequent calculations, this paper uses a regression-based filling method to fill missing values. The regression-based filling method is a data model-based filling method that can build a model based on known data to predict and fill missing values, thereby ensuring data integrity. Specifically, this method analyzes the correlation between data and uses a regression model to estimate reasonable values for the missing data, ensuring that the statistical properties of the filled data are consistent with those of the original data.

[0048] After filling missing values, standard deviation normalization is used to normalize the filled operating status data to eliminate dimensional differences between the data. Standard deviation normalization is a common data normalization technique that converts data of different dimensions and orders of magnitude into dimensionless relative values, making the data comparable. Normalized data typically ranges from 0 to 1, or is standardized with a mean of 0 and a standard deviation of 1.

[0049] After the above preprocessing steps, the obtained server operation status data (including system status data and hardware status data) will be used for subsequent analysis.

[0050] S2: Use the isolation forest algorithm on the preprocessed operating status data to construct the server's operating status isolation tree. According to the over-limit and under-limit conditions in the operating status data of each dimension of the server, determine the neighbor monitoring over-limit sequence and the neighbor monitoring under-limit sequence. Analyze the abnormality of the neighbor monitoring over-limit sequence and the neighbor monitoring under-limit sequence to construct the threshold deviation index.

[0051] It should be noted that the parameters and default values for the Isolation Forest algorithm are as follows: the default value for the number of isolated trees is 100, the default value for the isolation tree depth is 10, and the number of running status data points selected each time an isolation tree is constructed is 256. Implementers can select these parameters based on their specific circumstances. The process of constructing isolation trees in the Isolation Forest algorithm is well known and will not be further described here.

[0052] In server operation status monitoring, over-limit refers to the server operation data in each dimension exceeding the preset upper limit, and under-limit refers to the server operation data in each dimension falling below the preset lower limit.

[0053] Taking any moment as the target moment, the data of each dimension parameter corresponding to the target moment at a preset number of moments before the target moment is used as the neighbor monitoring sequence; obtaining the monitoring data when the server is running smoothly to determine the normal range of each dimension data when running smoothly;

[0054] The data of each dimension in the nearest neighbor monitoring sequence is compared with the maximum and minimum values in the normal interval range, and the difference between the data of each dimension and the maximum value constitutes the nearest neighbor monitoring over-limit sequence, in which the data less than 0 is set to 0; the difference between the data of each dimension and the minimum value constitutes the nearest neighbor monitoring lower limit sequence, in which the data greater than 0 is set to 0.

[0055] Threshold shift index, including:

[0056] The length of continuous appearance of non-zero data in the neighbor monitoring over-limit sequence and the neighbor monitoring under-limit sequence is recorded as the continuous length, and the interval where the non-zero data is located is recorded as the continuous interval;

[0057] Taking any moment as the target moment, traverse and calculate the product of the continuous length corresponding to the continuous interval in all neighboring monitoring out-of-limit sequences of each dimension data at the target moment and the sum of all data in the corresponding continuous interval, and sum the product results of all intervals as the out-of-limit anomaly cumulative index;

[0058] Traverse and calculate the product of the continuous length corresponding to the continuous interval in all the neighbor monitoring lower limit sequences of each dimension of the target time and the sum of all the data in the corresponding continuous interval, and sum the product results of all intervals as the lower limit anomaly cumulative index;

[0059] The over-limit anomaly cumulative index and the lower-limit anomaly cumulative index are added together as the threshold deviation index of each dimension data at the target moment.

[0060] For example, in this embodiment, the preset previous moments are 10 moments before the target moment, and the implementer can select according to the specific situation.

[0061] Specifically, the threshold shift index satisfies the following relationship:

[0062] ;

[0063] Where, Indicates that the server is running Moment The threshold deviation index of the dimension data, 、 Respectively represent the server running process Moment The number of consecutive intervals in the neighbor monitoring over-limit sequence and the neighbor monitoring under-limit sequence of the dimension data, Indicates the first The continuous length of the continuous interval, Indicates the first The sum of all data in a continuous interval, Indicates the first The continuous length of the continuous interval, Indicates the first The sum of all data in a continuous interval.

[0064] That is to say, during the operation of the server, the longer the continuous length of the monitoring data of one dimension exceeds or falls below the normal operating range, the or The larger it is, the more it exceeds or falls below the normal operating range, i.e. or The larger the value is, the more serious the abnormal operation state is, and therefore the larger the calculated threshold deviation index is.

[0065] To further clarify, failures in a server cluster are typically not isolated but rather result from multiple failures occurring simultaneously or as a chain reaction triggered by a single failure. The correlation between these failures primarily stems from temporal sequencing and the physical connectivity between servers or between components within them. For example, under normal server operation, operational data typically remains within a relatively stable range. For example, CPU utilization typically remains relatively stable under normal operation. For example, when a web server processes routine user requests, CPU utilization may fluctuate between 30% and 50%. When a server handles a large number of concurrent requests or runs complex computing tasks, CPU utilization increases significantly. For example, when a database server executes large-scale data queries or complex transactions, CPU utilization may approach 100%. Prolonged high CPU utilization can lead to slow server response times or even freezes or crashes. If this high utilization persists for an extended period, it can cause server performance bottlenecks, impacting the user experience.

[0066] This highlights the correlation, chain reaction, and complexity of failures in a server group. To effectively diagnose and resolve failures, quickly locate the root cause, prevent the spread of failures, and ensure stable server operation, the specific steps are as follows:

[0067] S3: Based on the threshold deviation index, the correlation characteristics between the data of each dimension during the server operation are analyzed, and a fault correlation index is constructed to measure the degree of fault correlation between data of different dimensions. Based on the fault correlation index, the difference in the degree of data anomaly between leaf nodes and child nodes in the isolation tree is analyzed to construct an intra-layer isolation index.

[0068] The calculation method of the fault correlation index includes:

[0069] Get the number of dimensions of the server's running status, take any dimension data as the target data, and sequentially traverse and calculate the sum of the threshold offset indices between the target data and the other dimension data except the target data, as well as the absolute difference between the threshold offset indices between the target data and the other dimension data except the target data, and add 1 to the absolute difference as the adjustment factor;

[0070] The sum of the ratios of the sum of the threshold deviation indices and the adjustment factor is taken as the fault correlation index at each moment during the server operation.

[0071] Specifically, the fault correlation index satisfies the following relationship:

[0072] ;

[0073] Where, Indicates that the server is running The fault correlation index at time The number of dimensions that represent the server's running status, Indicates that the server is running Moment The threshold deviation index of the dimension data, Indicates that the server is running Moment The threshold shift index of the dimension data, 1 represents the hyperparameter.

[0074] That is, the number of dimensions in the present invention is set to 8, which can be selected according to the situation. A hyperparameter is used in the denominator to avoid the denominator being zero and causing calculation failure, which can be selected according to the situation.

[0075] To further illustrate, in an isolation tree constructed based on server operating status data, if the isolation tree effectively isolates abnormal server operating status data, abnormal data will typically be isolated in the isolation tree's leaf nodes, while normal data will be assigned to the isolation tree's non-leaf nodes. This means that the data in the leaf nodes of the isolation tree is generally more abnormal, while the data in the child nodes of the non-leaf nodes is relatively less abnormal.

[0076] Intra-layer isolation indices include:

[0077] Taking any layer in the isolation tree as the layer data to be analyzed, calculate the absolute difference between the mean of the fault correlation index of the server running status data in the leaf nodes in the layer data to be analyzed and the mean of the fault correlation index of the server running status data in the non-leaf nodes in the layer data to be analyzed, and use the absolute difference as the mean difference between the nodes;

[0078] Calculate the absolute difference between the original segmentation threshold and the Otsu threshold of the server operation status of the layer data to be analyzed in the feature segmentation dimension, and add 1 to the absolute difference as the threshold adjustment factor;

[0079] The ratio between the mean difference between nodes and the threshold adjustment factor is used as the intra-layer isolation index of the layer data to be analyzed in the isolation tree.

[0080] Specifically, the intra-layer isolation index satisfies the following relationship:

[0081] ;

[0082] Where, Indicates the number of isolated trees The intra-layer isolation index of the layer, Indicates the number of isolated trees The mean value of the fault correlation index of the server operation status data in the leaf node layer, Indicates the number of isolated trees The mean value of the fault correlation index of the server operation status data in the non-leaf nodes of the layer, Indicates the number of isolated trees The original segmentation threshold of the layer server running status data in the feature segmentation dimension, Indicates the number of isolated trees Otsu threshold for layer server running status data on the feature segmentation dimension.

[0083] That is, in the isolated tree In the layer, if the difference in abnormality between the server operation status data in the leaf node and the child node is greater, that is, The larger it is, the smaller the difference between the original segmentation threshold of the data in the layer and the Otsu threshold is, that is, The smaller it is, the better the segmentation effect of the layer on abnormal data is, that is, the data with a greater degree of abnormality falls into the leaf nodes, and the normal or less abnormal data falls into the non-leaf nodes and enters the next layer for further segmentation. Therefore, the larger the calculated intra-layer isolation index is.

[0084] In the isolation tree constructed based on the server operation status data, the feature segmentation dimension of the data in the layer to be analyzed refers to the data segmentation dimension selected at that layer. The value of the server operation status data in that layer on the feature segmentation dimension is used as input, and the segmentation threshold is calculated using the Otsu threshold segmentation method. The Otsu threshold segmentation method is used to determine the optimal segmentation point and divide the data into different child nodes. This process is a well-known technology and will not be described in detail here. In order to avoid the denominator being zero and causing calculation failure, it can be selected according to the situation.

[0085] Further analysis shows that the intra-layer isolation index of each layer in the isolation tree reflects the segmentation effect of each layer in the isolation tree on the server abnormal data. If the entire isolation tree has a good recognition effect on the server abnormal data, the data with a larger degree of abnormality will be recognized by the isolation tree as early as possible, and the data with a larger degree of abnormality will fall into the leaf nodes with a shallower layer, and the data with a smaller degree of abnormality will fall into the leaf nodes with a deeper layer. That is, as the depth of the isolation tree increases, the degree of abnormality of the operating status data has a downward trend. The segmentation effect of the entire isolation tree on the abnormal data can be evaluated, and then the abnormal isolation weight of the isolation tree for the abnormal data can be determined. The specific method is as follows:

[0086] S4: Calculate the abnormal isolation weight of the isolated tree based on the intra-layer isolation index, take the ratio of the abnormal isolation weight of each isolated tree to the sum of the abnormal isolation weights of all isolated trees in the isolation forest as the significant abnormal weight, and improve the isolation forest algorithm based on the significant abnormal weight to diagnose server failures.

[0087] Abnormal isolated weights, including:

[0088] Taking any layer in the isolation tree as the layer data to be analyzed, the ratio between the isolation index of the layer above the layer data to be analyzed and the isolation index of the layer data to be analyzed is summed to obtain the abnormal isolation weight of the isolation book.

[0089] Specifically, the anomaly isolation weight satisfies the following relationship:

[0090] ;

[0091] Where, Indicates the The abnormal isolation weight of the isolated tree, Indicates the The depth of the isolated number, Indicates the number of isolated trees The intra-layer isolation index of the layer, Indicates the number of isolated trees The intra-layer isolation index of a layer.

[0092] That is to say, as the depth of the isolation tree increases, the downward trend of the abnormality of the server operation status data becomes more obvious, that is, The larger it is, the better the isolation effect of the isolation tree on abnormal data is, so the larger the calculated abnormal isolation weight is.

[0093] The server's real-time operating status data is used as the input of the isolation forest algorithm. The significant isolation weight is used as the weight of the anomaly score of each isolated tree. The anomaly scores of all isolated trees in the isolation forest are weighted and summed to obtain a comprehensive anomaly score.

[0094] If the comprehensive anomaly score is greater than the fault threshold, it indicates that the server has a fault and should be repaired in time. Otherwise, if it is less than or equal to the fault threshold, it indicates that the server is normal.

[0095] Exemplarily, the fault threshold is 0.75, which can be adjusted according to specific circumstances.

[0096] The present invention also provides a server fault diagnosis system for a server group. Figure 2 As shown, the system includes a processor and a memory. The memory stores computer program instructions. When executed by the processor, the computer program instructions implement a server fault diagnosis method for a server group according to the first aspect of the present invention. The system also includes other components familiar to those skilled in the art, such as a communication bus and a communication interface. The configuration and functions of these components are well known in the art and are therefore not described in detail here.

[0097] It should be noted that those skilled in the art may make various modifications and improvements without departing from the scope of the present invention, and these modifications and improvements fall within the scope of protection of the present invention. Therefore, the scope of protection of the patent for this invention shall be based on the appended claims.

Claims

1. A server fault diagnosis method for a server group, characterized in that: include: Obtain the running status data of the server during operation and perform preprocessing; The isolation forest algorithm is used on the preprocessed operating status data to construct an isolation tree of the server's operating status. Based on the over-limit and under-limit conditions in the server's operating status data in each dimension, the neighbor monitoring over-limit sequence and the neighbor monitoring under-limit sequence are determined. The abnormality of the neighbor monitoring over-limit sequence and the neighbor monitoring under-limit sequence is analyzed to construct a threshold deviation index. Based on the threshold deviation index, the correlation characteristics between the data of each dimension during the server operation are analyzed, and a fault correlation index is constructed to measure the degree of fault correlation between data of different dimensions. Based on the fault correlation index, the difference in the degree of data anomaly between leaf nodes and child nodes in the isolation tree is analyzed to construct an intra-layer isolation index. The abnormal isolation weight of the isolated tree is calculated according to the intra-layer isolation index, and the ratio of the abnormal isolation weight of each isolated tree to the sum of the abnormal isolation weights of all isolated trees in the isolation forest is used as the significant abnormal weight. The improved isolation forest algorithm is used based on the significant abnormal weight to diagnose server failures.

2. A server fault diagnosis method for a server group according to claim 1, characterized in that: The above-limit and below-limit include: In server operation status monitoring, over-limit refers to the server operation data in each dimension exceeding the preset upper limit, and under-limit refers to the server operation data in each dimension falling below the preset lower limit.

3. A server fault diagnosis method for a server group according to claim 1, characterized in that: Determining the neighbor monitoring over-limit sequence and the neighbor monitoring under-limit sequence includes: Taking any moment as the target moment, the data of each dimension parameter corresponding to the target moment at a preset number of moments before the target moment is used as the neighbor monitoring sequence; obtaining the monitoring data when the server is running smoothly to determine the normal range of each dimension data when running smoothly; The data of each dimension in the nearest neighbor monitoring sequence is compared with the maximum and minimum values in the normal interval range, and the difference between the data of each dimension and the maximum value constitutes the nearest neighbor monitoring over-limit sequence, in which the data less than 0 is set to 0; the difference between the data of each dimension and the minimum value constitutes the nearest neighbor monitoring lower limit sequence, in which the data greater than 0 is set to 0.

4. A server fault diagnosis method for a server group according to claim 1, characterized in that: The threshold offset index includes: recording the length of continuous appearance of non-zero data in the neighbor monitoring over-limit sequence and the neighbor monitoring lower limit sequence as the continuous length, and recording the interval where the non-zero data is located as the continuous interval; taking any moment as the target moment, traversing and calculating the product between the continuous length corresponding to the continuous interval in all neighbor monitoring over-limit sequences of each dimensional data at the target moment and the sum of all data in the corresponding continuous interval, and summing the product results of all intervals as the over-limit anomaly cumulative index; traversing and calculating the product between the continuous length corresponding to the continuous interval in all neighbor monitoring lower limit sequences of each dimensional data at the target moment and the sum of all data in the corresponding continuous interval, and summing the product results of all intervals as the lower limit anomaly cumulative index; adding the over-limit anomaly cumulative index and the lower limit anomaly cumulative index as the threshold offset index of each dimensional data at the target moment.

5. A server fault diagnosis method for a server group according to claim 1, characterized in that: The calculation method of the fault correlation index includes: Get the number of dimensions of the server's running status, take any dimension data as the target data, and sequentially traverse and calculate the sum of the threshold offset indices between the target data and the other dimension data except the target data, as well as the absolute difference between the threshold offset indices between the target data and the other dimension data except the target data, and add 1 to the absolute difference as the adjustment factor; The sum of the ratios of the sum of the threshold deviation indices and the adjustment factor is taken as the fault correlation index at each moment during the server operation.

6. A server fault diagnosis method for a server group according to claim 1, characterized in that: The intra-layer isolation index includes: Taking any layer in the isolation tree as the layer data to be analyzed, calculate the absolute difference between the mean of the fault correlation index of the server running status data in the leaf nodes in the layer data to be analyzed and the mean of the fault correlation index of the server running status data in the non-leaf nodes in the layer data to be analyzed, and use the absolute difference as the mean difference between the nodes; Calculate the absolute difference between the original segmentation threshold and the Otsu threshold of the server operation status of the layer data to be analyzed in the feature segmentation dimension, and add 1 to the absolute difference as the threshold adjustment factor; The ratio between the mean difference between nodes and the threshold adjustment factor is used as the intra-layer isolation index of the layer data to be analyzed in the isolation tree.

7. A server fault diagnosis method for a server group according to claim 1, characterized in that: The abnormal isolation weight includes: Taking any layer in the isolation tree as the layer data to be analyzed, the ratio between the isolation index of the layer above the layer data to be analyzed and the isolation index of the layer data to be analyzed is summed to obtain the abnormal isolation weight of the isolation book.

8. A server fault diagnosis method for a server group according to claim 1, characterized in that: The improved isolation forest algorithm based on significant anomaly weights is used to diagnose server failures, including: The server's real-time operating status data is used as the input of the isolation forest algorithm. The significant isolation weight is used as the weight of the anomaly score of each isolated tree. The anomaly scores of all isolated trees in the isolation forest are weighted and summed to obtain a comprehensive anomaly score. If the comprehensive anomaly score is greater than the fault threshold, it indicates that the server has a fault and should be repaired in time. Otherwise, if it is less than or equal to the fault threshold, it indicates that the server is normal.

9. The server fault diagnosis method for a server group according to claim 1, characterized in that: Preprocessing the operating status data includes: The operating status data includes system status data and hardware status data. The system status data includes: CPU occupancy rate, memory occupancy rate, network utilization rate and disk I / O utilization rate, which is used to reflect the software operating status of the server; the hardware status data includes: server temperature, fan speed, server power and CPU temperature, which is used to monitor the hardware health of the server; The regression filling method is used to fill the missing values in the operating status data, and the standard deviation normalization method is used to normalize the filled operating status data to achieve data standardization and complete the preprocessing of the operating status data.

10. A server fault diagnosis system for a server group, characterized in that: include: A processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the server fault diagnosis method for a server group according to any one of claims 1 to 9 is implemented.

Images