Method for detecting network sub-health of distributed storage system

Abstract

The invention relates to a method for detecting network sub-health of a distributed storage system, belongs to the technical field of network detection, and solves the problem that network sub-health detection in the prior art is prone to misjudgment and missed judgment. The method comprises the following steps: regularly collecting monitoring data of a front-end service and a back-end network of the distributed storage system; periodically analyzing monitoring data of front-end services according to a plurality of service thresholds and a plurality of abnormal scene modes to obtain a first analysis result set; regularly analyzing monitoring data of a back-end network according to a plurality of network thresholds and a clustering algorithm to obtain a second analysis result set; and obtaining a detection result whether network sub-health exists or not by periodically calculating a correlation coefficient of the first analysis result set and the second analysis result set. The accuracy of network sub-health detection is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of network detection, and in particular to a method for detecting sub-health of a distributed storage system network. Background Art

[0002] In a distributed storage system, network sub-health problems refer to the situation where the processing capacity of the storage system drops sharply due to occasional and irregular network delays, packet loss, packet errors, packet disorder, congestion, and other problems on the network link. However, sub-health problems are not deterministic network anomalies or normal problems, and are very ambiguous. Therefore, it is very important to accurately detect sub-health problems.

[0003] Currently, the underlying network latency and packet loss rate are usually used as monitoring indicators. The pre-set thresholds are used to identify whether the monitoring indicators are normal, thereby determining whether the entire distributed storage system has network sub-health problems.

[0004] However, the existing method is prone to misjudgment. For example, when the network delay between distributed nodes increases by 3ms, it has no impact on some business applications, but for other business applications, the business may be basically unavailable. If only the threshold is used, this abnormality cannot be monitored. For bandwidth-based business systems, the network bandwidth can easily reach the physical upper limit of the network card. At this time, the packet loss rate is very high, but there is no obvious impact on business storage. If the threshold is used, it will be misjudged as network sub-health. Summary of the invention

[0005] In view of the above analysis, an embodiment of the present invention aims to provide a method for detecting sub-health of a distributed storage system network, so as to solve the problem that the existing network sub-health detection is prone to misjudgment and omission.

[0006] An embodiment of the present invention provides a method for detecting sub-health of a distributed storage system network, comprising the following steps:

[0007] Regularly collect monitoring data of the front-end business and back-end network of the distributed storage system;

[0008] Regularly analyzing the monitoring data of the front-end business according to multiple business thresholds and multiple abnormal scenario modes to obtain a first analysis result set; regularly analyzing the monitoring data of the back-end network according to multiple network thresholds and clustering algorithms to obtain a second analysis result set;

[0009] By periodically calculating the correlation coefficient between the first analysis result set and the second analysis result set, a detection result of whether there is network sub-health is obtained.

[0010] Based on the further improvement of the above method, the monitoring data of the front-end business monitors the block storage, object storage and file storage of each storage pool in the distributed storage system, takes each block storage volume, object storage bucket and file storage directory as a business monitoring object, and collects the IOPS, storage bandwidth and storage latency of each business monitoring object.

[0011] Based on the further improvement of the above method, the monitoring data of the back-end network monitors each server node in the distributed storage system, takes each network card in each server node as a network monitoring object, and collects the network delay, network packet number, network bandwidth, network packet loss rate, network error packet rate and network card status of each network monitoring object.

[0012] Based on the further improvement of the above method, the monitoring data of the front-end business is regularly analyzed according to multiple business thresholds and multiple abnormal scenario modes to obtain a first analysis result set, including:

[0013] In each business analysis cycle, identify whether there is data with a storage delay greater than a maximum delay threshold for any business monitoring object. If so, the first analysis result is abnormal. Otherwise, based on multiple business thresholds and multiple abnormal scenario modes, identify whether the timing data of IOPS and storage bandwidth of any business monitoring object matches any abnormal scenario mode to obtain respective identification results. Based on the identification results of multiple consecutive business analysis cycles, obtain the first analysis result. Put the first analysis result into the first analysis result set.

[0014] Based on the further improvement of the above method, the time series data of IOPS and storage bandwidth of any business monitoring object includes: the original time series data of IOPS and storage bandwidth of each business monitoring object during the business analysis period, and the smoothed time series data obtained by taking the average value of each original time series data in the same quantity.

[0015] Based on further improvements of the above method, multiple abnormal scenario modes include: a cliff-like decline mode, a continuous decline mode and a repeated decline mode; the cliff-like decline mode is that the decline compared with the previous data is greater than or equal to the second decline threshold, and the duration of the data after the decline is greater than or equal to the first time threshold; the continuous decline mode is that the adjacent data continues to decline but the decline is less than the second decline threshold, and when the data after the decline reaches stability, the total decline is greater than or equal to the first decline threshold, and the duration is greater than or equal to the first time threshold; the repeated decline mode is that the decline compared with the previous data is greater than or equal to the third decline threshold, the duration of the data after the decline is less than the second time threshold and then recovers, and the number of occurrences within the first time threshold is greater than or equal to the number threshold.

[0016] Based on the further improvement of the above method, according to the identification results of multiple consecutive business analysis cycles, a first analysis result is obtained, including:

[0017] By setting the recognition result that matches any abnormal scenario pattern to 1, otherwise setting the recognition result to 0, the recognition result sequence of IOPS and storage bandwidth of the same business monitoring object is obtained; if the number of 1s in the same position of the two recognition result sequences is greater than or equal to the number of 0s, and the Pearson correlation coefficients of the two recognition result sequences are greater than or equal to the correlation threshold, then the first analysis result is abnormal, otherwise the first analysis result is normal.

[0018] Based on the further improvement of the above method, the monitoring data of the backend network is regularly analyzed according to multiple network thresholds and clustering algorithms to obtain a second analysis result set, including:

[0019] In each network analysis cycle, identify whether the network card status of any network monitoring object is disabled. If so, the second analysis result is abnormal; otherwise, the DBSCAN clustering algorithm is used to cluster the network delay of each network monitoring object to identify whether the network delay classified as noise is greater than the network delay threshold, or the difference with the network delay threshold is less than the delay difference threshold. If so, the second analysis result is abnormal, otherwise the second analysis result is normal; the second analysis result is put into the second analysis result set.

[0020] Based on the further improvement of the above method, by periodically calculating the correlation coefficient between the first analysis result set and the second analysis result set, the detection result of whether there is network sub-health is obtained, including:

[0021] By setting the abnormality to 1 and the normality to 0, a first analysis result sequence and a second analysis result sequence are obtained; sequences of the same length are periodically taken out as two sequences to be detected, and when the number of 1s at the same position of the two sequences to be detected is greater than or equal to the number of 0s, it is identified whether the correlation coefficient of the two sequences to be detected is greater than or equal to the correlation threshold. If so, network sub-health exists, otherwise, network sub-health does not exist.

[0022] Based on a further improvement of the above method, whether the correlation coefficient of two sequences to be detected is greater than or equal to the correlation threshold is identified by calculating the Pearson correlation coefficient of the two sequences to be detected, and calculating multiple Pearson correlation coefficients after multiple translations of the time points of any sequence to be detected; if the number of Pearson correlation coefficients greater than or equal to the correlation threshold is greater than half, it is judged to be greater than or equal to the correlation threshold, otherwise it is judged to be less than the correlation threshold.

[0023] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0024] 1. Monitor the front-end business and back-end network simultaneously, and analyze the correlation between front-end and back-end abnormal events to eliminate minor network anomalies that do not affect the business, as well as network anomalies caused by changes in normal business models, thereby improving the accuracy of network sub-health detection.

[0025] 2. For front-end business, the matching and identification of multiple abnormal scenario patterns and the correlation between business indicators are combined to improve the accuracy of business damage identification; for the back-end network, the clustering algorithm is used to analyze dynamic noise points, supplemented by static network thresholds, and the combination of dynamic and static improves the accuracy of network anomaly identification; finally, when business damage and network anomalies are strongly correlated, network sub-health is identified, which solves the existing problems of missed judgment and misjudgment that are prone to occur, and improves the reliability and stability of the entire distributed storage system.

[0026] In the present invention, the above-mentioned technical solutions can also be combined with each other to achieve more preferred combination solutions. Other features and advantages of the present invention will be described in the subsequent description, and some advantages can become obvious from the description, or can be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained through the contents particularly pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings are only used for the purpose of illustrating specific embodiments and are not to be considered as limiting the present invention. In the entire drawings, the same reference symbols represent the same components;

[0028] Figure 1 The present invention is a flowchart of a method for detecting sub-health of a distributed storage system network in an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0030] A specific embodiment of the present invention discloses a method for detecting sub-health of a distributed storage system network. Figure 1 As shown, the following steps are included:

[0031] S1. Regularly collect monitoring data of the front-end business and back-end network of the distributed storage system;

[0032] S2. Regularly analyze the monitoring data of the front-end business according to multiple business thresholds and multiple abnormal scenario modes to obtain a first analysis result set; regularly analyze the monitoring data of the back-end network according to multiple network thresholds and clustering algorithms to obtain a second analysis result set;

[0033] S3. Obtain a detection result of whether there is network sub-health by regularly calculating the correlation coefficient between the first analysis result set and the second analysis result set.

[0034] During implementation, the front-end business and back-end network are monitored simultaneously, and by analyzing the correlation between front-end and back-end abnormal events, minor network anomalies that do not affect the business and network anomalies caused by changes in normal business models are eliminated, thereby improving the accuracy of network sub-health detection.

[0035] It should be noted that the distributed storage system includes multiple storage nodes and server nodes, where the storage nodes store front-end business data by creating storage pools and different types of storage services, including block storage, object storage, and file storage; the server nodes interact with data by building a back-end physical network, and each server node includes at least one network card. A monitoring agent is deployed on each node, which samples and reports to the time series database according to the data sampling interval set in the configuration file. Exemplarily, the sampling interval is 3 seconds. If the disk space where the time series database is located is large enough, the sampling interval can be reduced to 1 second to improve data accuracy and make data analysis more accurate. The time series database sets the default storage time of historical data through configuration, and automatically cleans up after the storage time is reached.

[0036] It should be noted that, in step S1, the monitoring agent monitors the front-end business and the back-end network, that is, collects different monitoring indicators for each storage node and each server node.

[0037] Specifically, the front-end business monitoring objects include each block storage volume, object storage bucket, and file storage directory in each storage pool in all storage nodes, and the business monitoring indicators include IOPS (Input / Output Operations Per Second), storage bandwidth, and storage latency of each business monitoring object. These indicator data can be obtained by calling the API of each storage service.

[0038] The backend network monitoring objects include each network card in each server node, and the network monitoring indicators include network latency, number of network sent and received packets, network bandwidth, network packet loss rate, network error rate, and network card status for each network monitoring object. These indicator data can be analyzed from the returned results after executing system commands.

[0039] For example, the ping command can be used to test the round-trip time of the network and calculate the latency; when the server is a Linux system, the ifconfig or ip command can be used to view the statistics of the network interface, including the number of packets sent and received; or the ethtool command can be used to view more detailed network interface statistics, such as the number of lost packets and error packets.

[0040] Furthermore, step S2 is divided into step S21 of analyzing the collected monitoring data of the front-end service, and step S22 of analyzing the collected monitoring data of the back-end network.

[0041] It should be noted that although the changes in IOPS and storage bandwidth are strongly related to business pressure and business scenarios, the sudden drop in these two indicators may also be caused by network sub-health problems; storage latency is a relatively stable indicator. During normal operation of different businesses, the latency is usually within a certain range and the fluctuation is very small; a sudden increase in latency is likely to be caused by network sub-health problems. Therefore, when performing business monitoring data analysis in step S21, first identify the change in storage latency, and when the storage latency is within the normal range, then identify the change in IOPS and storage bandwidth.

[0042] Specifically, the monitoring data of the front-end business is regularly analyzed according to multiple business thresholds and multiple abnormal scenario modes to obtain a first analysis result set, including:

[0043] In each business analysis cycle, identify whether there is data with a storage delay greater than a maximum delay threshold for any business monitoring object. If so, the first analysis result is abnormal. Otherwise, based on multiple business thresholds and multiple abnormal scenario modes, identify whether the timing data of IOPS and storage bandwidth of any business monitoring object matches any abnormal scenario mode to obtain respective identification results. Based on the identification results of multiple consecutive business analysis cycles, obtain the first analysis result. Put the first analysis result into the first analysis result set.

[0044] It should be emphasized that in order to reduce the random fluctuations in the original time series data actually collected and make the potential trends and patterns more obvious, when identifying whether the time series data matches any abnormal scenario mode, both the original time series data of IOPS and storage bandwidth of each business monitoring object during the business analysis cycle are analyzed, and the smoothed time series data obtained by taking the average of the same number of original time series data are analyzed.

[0045] It should be noted that taking the average of the same number includes: taking the average of the same number of non-overlapping numbers in sequence, for example, calculating an average for every 3 data; and taking the average of the data in the window in a sliding window manner, for example, the length of the sliding window is 3 and the sliding step is 1.

[0046] Furthermore, in this embodiment, three abnormal scenario modes are mainly identified for IOPS and storage bandwidth: a cliff-like drop mode, a continuous drop mode, and a repeated drop mode. If the amount of change in data increase or decrease is within the change threshold range, the data is considered to be in a stable state.

[0047] Specifically, the cliff-like decline mode is characterized by a sudden drop in data that lasts for a long time, that is, the current data drops by a magnitude greater than or equal to a second decline threshold compared with the previous data, and the duration of the data after the decline is greater than or equal to the first time threshold; exemplarily, when the second decline threshold is 30% and the first time threshold is 120 seconds, the data drops by a magnitude greater than or equal to 30% of the previous data, and the duration is greater than or equal to 120 seconds.

[0048] The characteristic of the continuous decline mode is that there is a continuous downward trend and finally stabilizes at a very low level, that is, the data continues to decline compared with adjacent data but the decline is less than the second decline threshold. When the data after the decline is in a stable state, the total decline is greater than or equal to the first decline threshold, and the duration is greater than or equal to the first time threshold; exemplarily, when the first decline threshold is 70%, the value that continues to decline to the stable state is 70% of the data before the decline, and the duration is greater than or equal to 120 seconds.

[0049] The repeated decline mode is characterized by a sudden decline that lasts for a very short time and then recovers, and there are multiple sudden drop points that are repeated repeatedly, that is, the decline of the current data compared with the previous data is greater than or equal to the third decline threshold, the data after the decline lasts for less than the second time threshold and then recovers, and the number of occurrences within the first time threshold is greater than or equal to the number threshold; exemplarily, the third decline threshold is 20%, the second time threshold is 10 seconds, and the number threshold is 3, the data after each decline is 20% of the data before the decline, lasts for less than 10 seconds and then recovers to the data before the decline, and it occurs at least 3 times within 120 seconds.

[0050] Preferably, the first time threshold is greater than the second time threshold, the first falling threshold is greater than the second falling threshold, and the second falling threshold is greater than the third falling threshold.

[0051] This embodiment takes into account that the fluctuation of the indicator may be a change in the business model, such as an IOPS-type business suddenly becoming a bandwidth-type business, or the IO size of an IOPS-type business changing irregularly. In order to accurately identify whether the business is damaged, the final result is obtained through multiple rounds of analysis and put into the first analysis result set.

[0052] Specifically, according to the identification results of multiple consecutive business analysis cycles, a first analysis result is obtained, including:

[0053] By setting the recognition result that matches any abnormal scenario pattern to 1, otherwise setting the recognition result to 0, the recognition result sequence of IOPS and storage bandwidth of the same business monitoring object is obtained; if the number of 1s in the same position of the two recognition result sequences is greater than or equal to the number of 0s, and the Pearson correlation coefficients of the two recognition result sequences are greater than or equal to the correlation threshold, then the first analysis result is abnormal, otherwise the first analysis result is normal.

[0054] It should be noted that when the number of 1s at the same position of two recognition result sequences is greater than or equal to 0, the numerical range of the Pearson correlation coefficient is [-1,1]. When it is 0, it means there is no linear correlation, greater than 0 means positive correlation, and less than 0 means negative correlation. When both IOPS and storage bandwidth decrease, the probability of business damage is relatively high. Therefore, this embodiment classifies the Pearson correlation coefficient greater than 0: when the correlation coefficient is 0.8 to 1.0, it is extremely correlated; when it is 0.6 to 0.8, it is strongly correlated; when it is 0.4 to 0.6, it is moderately correlated; when it is 0.2 to 0.4, it is weakly correlated; and when it is 0 to 0.2, it is extremely weakly correlated or uncorrelated. Preferably, the correlation threshold is set to 0.7. That is, when the Pearson correlation coefficient of the recognition result sequence of IOPS and storage bandwidth of the same business monitoring object is greater than 0.7, it means that the two are strongly correlated. At this time, the business is damaged and the first analysis result is abnormal. Otherwise, the first analysis result is normal.

[0055] In the prior art, only the threshold is usually used to judge the network delay. If the threshold is configured too low, it may have affected the business, and it is easy to miss the judgment; if the threshold is configured too high, it is easy to make a wrong judgment when the business pressure is high. In order to solve the problems of missed judgment and wrong judgment, this embodiment mainly uses the clustering algorithm to analyze the dynamic noise points, supplemented by the static network threshold, and combines the dynamic and static to improve the accuracy of network anomaly identification.

[0056] Furthermore, in the network monitoring indicators, the network card status is enabled when it is normal and disabled when it is abnormal. Once the disabled status appears, there is no need to judge other indicators. Therefore, the network card status is identified first. When the network card status is normal, the number of network packets received and sent, network bandwidth, network packet loss rate and network packet error rate will be reflected in the network delay indicator once abnormalities occur. Therefore, in order to improve the analysis speed, in step S22, the monitoring data of the back-end network is analyzed with the network card status and network delay as the main analysis indicators.

[0057] Specifically, the monitoring data of the backend network is regularly analyzed according to multiple network thresholds and clustering algorithms to obtain a second analysis result set, including:

[0058] In each network analysis cycle, identify whether the network card status of any network monitoring object is disabled. If so, the second analysis result is abnormal; otherwise, the DBSCAN clustering algorithm is used to cluster the network delay of each network monitoring object to identify whether the network delay classified as noise is greater than the network delay threshold, or the difference with the network delay threshold is less than the delay difference threshold. If so, the second analysis result is abnormal, otherwise the second analysis result is normal; the second analysis result is put into the second analysis result set.

[0059] It should be noted that when the DBSCAN clustering algorithm is used, the settings of the neighborhood radius (eps) and the minimum number of points (min_samples) directly affect the analysis results. After a large number of experimental tests, the neighborhood radius in this embodiment is set to 1.5 times the normal value of the network delay, and the minimum number of points is 1 / 3 of the number of network delay samples. After clustering, the network delay data is divided into several clusters and noise (data that does not belong to any cluster). In this embodiment, the network delay classified as noise is used as data that may cause network anomalies. These data are further compared with the network delay threshold. If any network delay is greater than the network delay threshold, or any difference obtained by comparison is less than the delay difference threshold, that is, very close to the network delay threshold, the second analysis result is abnormal, and the corresponding network monitoring object (network card) and its network monitoring indicators are recorded, otherwise the second analysis result is normal.

[0060] Through step S2, the first analysis result set of the front-end business and the second analysis result set of the back-end network are respectively obtained. Next, in step S3, the two analysis result sets are comprehensively analyzed. By analyzing the correlation between "business damage" and "network anomaly", the network anomaly that will cause business damage is analyzed, which greatly improves the accuracy of judging network sub-health problems.

[0061] Specifically, by periodically calculating the correlation coefficient between the first analysis result set and the second analysis result set, the detection result of whether there is network sub-health is obtained, including:

[0062] By setting the abnormality to 1 and the normality to 0, a first analysis result sequence and a second analysis result sequence are obtained; sequences of the same length are periodically taken out as two sequences to be detected, and when the number of 1s at the same position of the two sequences to be detected is greater than or equal to the number of 0s, it is identified whether the correlation coefficient of the two sequences to be detected is greater than or equal to the correlation threshold. If so, network sub-health exists, otherwise, network sub-health does not exist.

[0063] It should be noted that, based on the comprehensive detection cycle, sequences with the same length before each detection time point are respectively taken from the first analysis result sequence and the second analysis result sequence as two sequences to be detected; considering that the network anomaly usually occurs first and the service is damaged later, there is a certain time difference between the first analysis result sequence and the second analysis result sequence. In this embodiment, data is shifted and the detection results of multiple correlation coefficients are integrated to ensure the accuracy of the final detection result.

[0064] Specifically, to identify whether the correlation coefficient of two sequences to be detected is greater than or equal to the correlation threshold, the Pearson correlation coefficient of the two sequences to be detected is calculated, and multiple Pearson correlation coefficients are calculated after multiple translations of the time points of any sequence to be detected; if the number of Pearson correlation coefficients greater than or equal to the correlation threshold is greater than half, it is judged to be greater than or equal to the correlation threshold, and the largest Pearson correlation coefficient is taken as the final correlation coefficient; otherwise, it is judged to be less than the correlation threshold.

[0065] It should be noted that the time point shift of the sequence to be detected is to shift the sequence to be detected corresponding to the service forward, or to shift the sequence to be detected corresponding to the network backward, and the number of shifts can be set by pre-configuration. Exemplarily, the backward shift of the sequence to be detected corresponding to the network means to shift the data at time point 1 backward to time point 2, and the same applies to other data.

[0066] It should be noted that the multiple business thresholds and multiple network thresholds used in this embodiment are determined according to the storage medium, network architecture and network type actually used by the distributed storage system. Each threshold can be configured as a specific value, and then a change threshold or a difference threshold is configured to achieve identification within the threshold range, or it can be directly configured as a range.

[0067] Compared with the prior art, the method for detecting sub-health of a distributed storage system network provided in this embodiment monitors the front-end business and the back-end network simultaneously, and excludes minor network anomalies that do not affect the business, as well as network anomalies caused by changes in normal business models by analyzing the correlation between front-end and back-end abnormal events, thereby improving the accuracy of network sub-health detection; the front-end business is combined with the matching and identification of multiple abnormal scenario patterns and the correlation between business indicators to improve the accuracy of business damage identification; the back-end network is mainly analyzed by clustering algorithm to identify dynamic noise points, supplemented by static network thresholds, and the accuracy of network anomaly identification is improved by combining dynamic and static methods; finally, when business damage and network anomalies are strongly correlated, network sub-health is identified, solving the existing problems of missed judgment and misjudgment that are prone to occur, and improving the reliability and stability of the entire distributed storage system.

[0068] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0069] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for detecting sub-health of a distributed storage system network, characterized in that: The following steps are involved: Regularly collect monitoring data of the front-end business and back-end network of the distributed storage system; Regularly analyzing the monitoring data of the front-end business according to multiple business thresholds and multiple abnormal scenario modes to obtain a first analysis result set; regularly analyzing the monitoring data of the back-end network according to multiple network thresholds and clustering algorithms to obtain a second analysis result set; By periodically calculating the correlation coefficient between the first analysis result set and the second analysis result set, a detection result of whether network sub-health exists is obtained.

2. The method for detecting sub-health of a distributed storage system network according to claim 1, characterized in that: The monitoring data of the front-end business monitors the block storage, object storage and file storage of each storage pool in the distributed storage system, takes each block storage volume, object storage bucket and file storage directory as a business monitoring object, and collects the IOPS, storage bandwidth and storage latency of each business monitoring object.

3. The method for detecting sub-health of a distributed storage system network according to claim 1, characterized in that: The monitoring data of the back-end network monitors each server node in the distributed storage system, takes each network card in each server node as a network monitoring object, and collects network latency, network number of sent and received packets, network bandwidth, network packet loss rate, network error packet rate and network card status of each network monitoring object.

4. The method for detecting sub-health of a distributed storage system network according to claim 2, characterized in that: The method of regularly analyzing the monitoring data of the front-end business according to multiple business thresholds and multiple abnormal scenario modes to obtain a first analysis result set includes: In each business analysis cycle, identify whether there is data with a storage delay greater than a maximum delay threshold for any business monitoring object. If so, the first analysis result is abnormal. Otherwise, based on multiple business thresholds and multiple abnormal scenario modes, identify whether the timing data of IOPS and storage bandwidth of any business monitoring object matches any abnormal scenario mode to obtain respective identification results. Based on the identification results of multiple consecutive business analysis cycles, obtain the first analysis result. Put the first analysis result into the first analysis result set.

5. The method for detecting sub-health of a distributed storage system network according to claim 4, characterized in that: The time series data of IOPS and storage bandwidth of any business monitoring object includes: the original time series data of IOPS and storage bandwidth of each business monitoring object in the business analysis period, and the smoothed time series data obtained by taking the average value of each original time series data in the same quantity.

6. The method for detecting sub-health of a distributed storage system network according to claim 4, characterized in that: The multiple abnormal scenario modes include: a cliff-like decline mode, a continuous decline mode and a repeated decline mode; the cliff-like decline mode is that the decline amplitude compared with the previous data is greater than or equal to the second decline threshold, and the duration of the data after the decline is greater than or equal to the first time threshold; the continuous decline mode is that the adjacent data continues to decline but the decline amplitude is less than the second decline threshold, and when the data after the decline reaches stability, the total decline amplitude is greater than or equal to the first decline threshold, and the duration is greater than or equal to the first time threshold; the repeated decline mode is that the decline amplitude compared with the previous data is greater than or equal to the third decline threshold, the duration of the data after the decline is less than the second time threshold and then recovers, and the number of occurrences within the first time threshold range is greater than or equal to the number threshold.

7. The method for detecting sub-health of a distributed storage system network according to claim 4, characterized in that: According to the identification results of multiple consecutive business analysis cycles, a first analysis result is obtained, including: By setting the recognition result that matches any abnormal scenario pattern to 1, otherwise setting the recognition result to 0, the recognition result sequence of IOPS and storage bandwidth of the same business monitoring object is obtained; if the number of 1s in the same position of the two recognition result sequences is greater than or equal to the number of 0s, and the Pearson correlation coefficients of the two recognition result sequences are greater than or equal to the correlation threshold, then the first analysis result is abnormal, otherwise the first analysis result is normal.

8. The method for detecting sub-health of a distributed storage system network according to claim 3, characterized in that: The step of regularly analyzing the monitoring data of the backend network according to the multiple network thresholds and the clustering algorithm to obtain a second analysis result set includes: In each network analysis cycle, identify whether the network card status of any network monitoring object is disabled. If so, the second analysis result is abnormal; otherwise, the DBSCAN clustering algorithm is used to cluster the network delay of each network monitoring object to identify whether the network delay classified as noise is greater than the network delay threshold, or the difference with the network delay threshold is less than the delay difference threshold. If so, the second analysis result is abnormal, otherwise the second analysis result is normal; the second analysis result is put into the second analysis result set.

9. The method for detecting sub-health of a distributed storage system network according to claim 1, characterized in that: The method of obtaining the detection result of whether there is network sub-health by regularly calculating the correlation coefficient between the first analysis result set and the second analysis result set includes: By setting the abnormality to 1 and the normality to 0, a first analysis result sequence and a second analysis result sequence are obtained; sequences of the same length are periodically taken out as two sequences to be detected, and when the number of 1s at the same position of the two sequences to be detected is greater than or equal to the number of 0s, it is identified whether the correlation coefficient of the two sequences to be detected is greater than or equal to the correlation threshold. If so, network sub-health exists, otherwise, network sub-health does not exist.

10. The method for detecting sub-health of a distributed storage system network according to claim 9, characterized in that: The identification of whether the correlation coefficient of the two sequences to be detected is greater than or equal to the correlation threshold is performed by calculating the Pearson correlation coefficient of the two sequences to be detected, and calculating multiple Pearson correlation coefficients after multiple translations of the time points of any sequence to be detected; if the number of Pearson correlation coefficients greater than or equal to the correlation threshold is greater than half, it is determined to be greater than or equal to the correlation threshold, otherwise it is determined to be less than the correlation threshold.

Images