A distributed storage node failure detection system

Abstract

The application provides a distributed storage node fault detection system, and relates to the technical field of distributed storage, which comprises a data acquisition module configured to acquire node running state data, a feature extraction module configured to perform feature calculation on the node running state data to obtain a feature vector, a distributed anomaly detection module adopting a three-layer detection architecture, including a local layer, a regional layer and a global layer, a fault diagnosis module configured to receive the anomaly detection result of the global layer, perform fault classification and generate a fault diagnosis report, and an early warning decision module configured to perform response strategy matching through a preset strategy template and a dynamically generated rule according to the fault diagnosis report, perform strategy priority sorting, and select the preset strategy with the highest priority as the final recommended strategy. The application can timely find the abnormal state of the storage node, accurately identify the fault type, quickly locate the fault root cause, and provide intelligent early warning decision support.

Description

Technical Field

[0001] The present invention relates to the field of distributed storage technology, and in particular to a distributed storage node fault detection system. Background Art

[0002] As distributed storage systems continue to expand, the number of nodes is growing exponentially, and system architectures are becoming increasingly complex. Frequent failures of various types have become a significant factor impacting system reliability. Currently, mainstream distributed storage systems generally use threshold-based fault detection methods. These methods set fixed thresholds for key metrics such as CPU utilization, memory usage, and I / O performance, triggering alerts when these metrics exceed the preset thresholds. This approach is simple to implement and deploy, and performs well in small-scale systems.

[0003] With the advancement of fault detection technology, researchers have proposed machine learning-based fault detection methods. These methods collect system operation data, extract multi-dimensional features such as time and frequency domain characteristics, and use algorithms such as cluster analysis and anomaly detection to model system states and identify abnormal conditions. Compared to traditional threshold detection methods, machine learning methods have stronger adaptability and can better handle complex fault scenarios.

[0004] However, existing fault detection methods still have several technical flaws: First, fixed-threshold detection methods struggle to adapt to dynamic changes in system load, and are prone to false positives and missed negatives. Second, single-node fault detection cannot effectively identify complex faults caused by the coordinated efforts of multiple nodes. Finally, detection results lack interpretability, making it difficult to provide effective decision support for operations and maintenance personnel. These issues severely restrict the reliability and maintainability of distributed storage systems. Summary of the Invention

[0005] In view of this, the present invention proposes a distributed storage node fault detection system. By constructing a three-layer detection architecture with multi-module collaboration, it can achieve comprehensive monitoring of the operating status of storage nodes, precise identification of abnormal behaviors, accurate diagnosis of fault types, and intelligent decision-making of early warning strategies, thereby improving the reliability and maintainability of the distributed storage system and reducing operation and maintenance costs.

[0006] The technical solution of the present invention is achieved as follows:

[0007] The present invention provides a distributed storage node fault detection system, comprising:

[0008] a data collection module configured to collect node operation status data and perform preprocessing and standardization on the node operation status data;

[0009] A feature extraction module is configured to perform feature calculation on the node operation status data and perform feature dimension reduction through principal component analysis to obtain a feature vector;

[0010] The distributed anomaly detection module adopts a three-layer detection architecture, including local layer, regional layer, and global layer. The local layer is configured to perform real-time cluster analysis on feature vectors using a clustering algorithm, the regional layer is configured to fuse the cluster analysis results of the local layer, and the global layer is configured to perform anomaly detection on the fused information.

[0011] A fault diagnosis module is configured to receive anomaly detection results at the global layer, classify faults, perform feature association analysis based on association rule mining, locate the root cause using a location algorithm, and generate a fault diagnosis report;

[0012] The early warning decision module is configured to classify confirmed faults based on multi-level early warning thresholds according to the fault diagnosis report, set the early warning suppression time window, match the response strategy through preset strategy templates and dynamically generated rules, and use a weighted scoring mechanism to prioritize the strategies, and select the preset strategy with the highest priority as the final recommended strategy.

[0013] Based on the above technical solution, preferably, the node operation status data is divided into three categories according to the monitoring indicator type, namely system-level indicator data, storage-level indicator data and network-level indicator data, among which:

[0014] System-level metrics include CPU usage, memory usage, system load, and process status;

[0015] Storage-level metrics include I / O performance, disk usage, read / write bandwidth, and disk health status.

[0016] Network-level metrics include network latency, packet loss rate, bandwidth usage, and connection status.

[0017] Based on the above technical solution, preferably, the process of the data acquisition module performing preprocessing and standardization on the node operation status data includes:

[0018] The collected original data were processed for missing values, and the multivariate linear regression interpolation method was used to repair the data by combining temporal correlation and spatial correlation;

[0019] Adaptive median filtering is used to smooth outliers, and the window size is dynamically adjusted according to the degree of data fluctuation;

[0020] The Z-score method was used to standardize the data, and the skewness correction coefficient was introduced to correct the asymmetry of the data distribution.

[0021] On the basis of the above technical solution, preferably, when the feature extraction module performs feature calculation on the node operation status data, a multi-dimensional feature extraction method is adopted, by calculating the time domain features, frequency domain features, time-frequency features and high-order statistical features, and optimizing the extracted features; wherein the time domain features include first-order moments, second-order moments, third-order moments and fourth-order moments, the frequency domain features obtain the power spectrum density and frequency band energy distribution through fast Fourier transform, the time-frequency features are multi-scale decomposition based on the db4 wavelet basis, and the high-order statistical features extract nonlinear features through the ARIMA model.

[0022] Based on the above technical solution, preferably, the optimization processing of the extracted features includes:

[0023] Calculate the feature importance score SI(f i ), sort the features according to their importance scores, select the top N features to be retained, and the feature importance score SI(f i ) is calculated as follows:

[0024] SI(f i )=α1·IG(f i )+α2·GR(f i )+α3·Relief(f i )

[0025] Where, f i represents the i-th feature, IG(f i ) is the information gain value, GR(f i ) is the information gain rate, Relief(f i ) is the score of the Relief algorithm, α1, α2, and α3 are weight coefficients and satisfy α1+α2+α3=1;

[0026] The correlation between features is calculated based on the Pearson correlation coefficient, and highly correlated redundant features are eliminated by setting the correlation threshold θ:

[0027]

[0028] In the formula, R(f i ,f j ) is the feature f i With feature f j The feature correlation, Cov(f i ,f j ) is the feature f i With feature f j The covariance of and They are respectively features f i With feature f j The standard deviation of

[0029] Perform principal component analysis and dimensionality reduction on the retained features to obtain the reduced dimensionality feature vector F = {f′1,f′2,...,f′ k}, where k is the feature dimension after dimensionality reduction, and the k value is determined by calculating the cumulative variance contribution rate, so that the cumulative variance contribution rate of the first k principal components is not less than the preset threshold η.

[0030] Based on the above technical solution, preferably, in the distributed anomaly detection module, the clustering algorithm is executed as follows:

[0031] A1. Normalize the eigenvectors and map them to the interval [0,1]. Treat each eigenvector as a point and calculate the Euclidean distance d between the two points. ij ;

[0032] A2. Calculate the local density of each point:

[0033] ρ i,t =∑ j χ(d ij -d c )·τ(t)

[0034] Where, ρ i,t is the local density, χ is the indicator function, d c is the cutoff distance, τ(t)=e -λt is the time attenuation factor, λ is the attenuation coefficient, and t is the time interval;

[0035] A3. Calculate the minimum distance δ from each point to the high-density point i =min(d ij ), where j satisfies ρ j,t >ρ i,t ;

[0036] A4. Calculate the overall score:

[0037] γ i =ρ i,t ·δ i ·(1+ε·En(i))

[0038] Where, γ i is the comprehensive score value, En(i) is the local entropy value of point i, and ε is the entropy weight coefficient;

[0039] A5. Determine the local density threshold:

[0040] th den =μ·mean(ρ)+σ·std(ρ)

[0041] Where th denrepresents the local density threshold, μ is the mean adjustment coefficient, σ is the standard deviation adjustment coefficient, mean(ρ) is the mean value of the local density, and std(ρ) is the standard deviation of the local density;

[0042] Calculate the distance anomaly threshold:

[0043] th dis =κ1·d max +κ2·std(D)

[0044] Where th dis is the distance anomaly threshold, κ1 is the distance benchmark coefficient, κ2 is the fluctuation adjustment coefficient, d max is the maximum distance of the current cluster, std(D) is the standard deviation of the distance within the cluster;

[0045] A6. Based on the comprehensive score γ i Determine the cluster center according to the local density threshold th den Perform cluster division and use the distance anomaly threshold th dis Identify outliers;

[0046] A7. Based on the cluster division and outlier identification results, generate local clustering results, including cluster information {c,n c ,d max} and the set of outliers, the cluster information includes the cluster center c, the number of vectors in the cluster n c and the maximum distance d max .

[0047] Based on the above technical solution, preferably, the fusion processing of the cluster analysis results of the local layer includes:

[0048] B1, receiving the local clustering results of each node in the area, including the cluster center set C = {c1, c2, ..., c m}、N={n1,n2,...,n m}、Maximum distance set And the set of outliers A=a1,a2,...,a k ;

[0049] B2. Calculate the inter-cluster similarity matrix S, for any two cluster centers c x and c y , and its similarity is calculated as follows:

[0050]

[0051] Where, s(c x ,c y ) represents the cluster center c x and c yThe similarity of , σ is the Gaussian kernel parameter, determined by cross-validation; ‖c x -c y ‖ represents the Euclidean distance between the two cluster centers, n x ,n y represents the number of vectors in the two clusters, Indicates the maximum distance between two clusters;

[0052] B3. Construct a hierarchical clustering tree T and use the adaptive threshold method to merge clusters:

[0053] Merge th =β1×(mean(S)+β2×std(S))

[0054] In the formula, Merge th is the merging threshold, β1 is the benchmark coefficient, β2 is the variance adjustment coefficient, mean(S) is the mean of the similarity matrix S, and std(S) is the standard deviation of the similarity matrix S;

[0055] B4. Calculate the regional anomaly index RAI:

[0056] RAI(c)=w1×DS(c)+w2×IS(c)+w3×TS(c)+w4×AD(c)

[0057] Where DS(c) is the density score, which indicates the density of the cluster; IS(c) is the isolation score, which indicates the degree of isolation of the cluster from other clusters; TS(c) is the temporal correlation score, which indicates the temporal evolution characteristics of the cluster; AD(c) is the anomaly distribution feature, which reflects the distribution characteristics of the anomalies around the cluster; w1, w2, and w3 are weight coefficients, and they satisfy w1+w2+w3=1.

[0058] B5. Generate regional status descriptor RSD={C * ,N * ,D * ,RAI * ,T *}, where C * is the set of cluster centers after merging, N * is the updated set of vectors in the cluster, D * is the updated maximum distance set, RAI * is the set of abnormal indices, T * is the timestamp.

[0059] Based on the above technical solution, preferably, the process of performing anomaly detection at the global layer includes:

[0060] C1. Construct the global state matrix GSM = [RSD1; RSD2; ...; RSD r], where r is the total number of regions;

[0061] C2. Calculate the inter-region correlation strength matrix (RCM) and establish a regional correlation relationship diagram by analyzing the similarity of performance indicators, network topology distance, and the similarity of anomaly patterns:

[0062] RCM(a,b)=∈1×PS(a,b)+∈2×TS(a,b)+∈3×AS(a,b)

[0063] Where RCM(a,b) represents the correlation strength matrix between region a and region b, PS(a,b) is the performance index similarity between region a and region b, TS(a,b) is the topological distance similarity between region a and region b, AS(a,b) is the anomaly pattern similarity between region a and region b, ∈1, ∈2, ∈3 are weight coefficients, and ∈1+∈2+∈3=1;

[0064] C3. Based on the sliding time window mechanism, analyze the anomaly pattern in the spatiotemporal dimension, including tracking the evolution trend of the anomaly state, identifying the propagation path of the anomaly between regions, and predicting the possible spread direction of the anomaly;

[0065] C4. Comprehensively evaluate the degree of abnormality by weightedly fusing local abnormality features, regional abnormality features, and associated abnormality features to obtain the global abnormality score GAS.

[0066] C5. Adopting an adaptive threshold method to determine anomalies. The threshold is dynamically adjusted based on the statistical characteristics of historical anomaly data.

[0067] C6. Output the standardized anomaly detection result set ADR, which includes the global anomaly score GAS, the location information AL of the abnormal area, the quantitative index of the anomaly degree AQ, the propagation situation feature vector PF, the impact range prediction IR, and the timestamp T.

[0068] On the basis of the above technical solution, preferably, the fault diagnosis module receives the abnormal detection result set ADR as input and processes it through the following subunits:

[0069] The fault classification unit builds a multi-level classification model based on the XGBoost decision tree algorithm. It uses the global anomaly score GAS, the abnormal area feature AL, and the propagation feature vector PF as feature inputs. It trains the model based on the historical fault sample library to achieve probabilistic prediction of the fault type and outputs the fault type vector FT containing the confidence level.

[0070] The feature association analysis unit uses the Apriori algorithm to mine the association rules between abnormal patterns and fault types based on the time series evolution data of abnormal areas and the propagation relationship between regions, constructs the fault feature map G, and establishes the feature association strength matrix FCM for each fault type;

[0071] The root cause location unit takes the area where the anomaly first occurs as the starting point, combines the propagation situation feature PF and network topology information, uses the random walk algorithm to analyze the fault propagation path, uses the PageRank algorithm to evaluate the importance of nodes, and outputs the root cause probability ranking list RC and the fault impact range assessment report IR;

[0072] The diagnostic report generation unit integrates the fault type FT, feature correlation map G and root cause analysis results RC to generate a standardized diagnostic report DR that includes fault location, impact assessment and repair suggestions, and provides visual fault diagnosis results.

[0073] On the basis of the above technical solution, preferably, the early warning decision module processes the anomaly detection result set ADR and the fault diagnosis report DR through the following subunits:

[0074] The fault level assessment unit comprehensively analyzes the global anomaly score GAS, fault type probability distribution FT, and impact range prediction IR, establishes a multi-dimensional fault severity assessment system, and dynamically generates the fault level assessment result SL;

[0075] The early warning control unit implements a time window-based early warning trigger mechanism based on the fault level SL and the propagation situation feature PF, and determines the alarm sequence through the early warning priority score;

[0076] The strategy matching unit selects a matching solution from the preset strategy library based on the current fault type FT and the root cause analysis result RC;

[0077] The decision optimization unit performs multi-objective optimization on candidate processing strategies, comprehensively considering factors such as fault severity SL, impact range IR, processing cost and risk, and outputs the optimal strategy execution plan and execution sequence.

[0078] The present invention has the following beneficial effects compared to the prior art:

[0079] (1) The present invention implements anomaly detection by constructing five collaborative functional modules: data acquisition, feature extraction, distributed anomaly detection, fault diagnosis, and early warning decision-making. It uses a three-layer detection architecture to implement anomaly detection and combines machine learning algorithms for fault classification and root cause location. This achieves full automation of the entire process from data acquisition and anomaly detection to fault diagnosis. The system can promptly detect abnormal states of storage nodes, accurately identify fault types, quickly locate the root causes of faults, and provide intelligent early warning decision-making support, thereby improving the reliability and maintainability of distributed storage systems.

[0080] (2) During the feature extraction process, the present invention calculates time domain features, frequency domain features, time-frequency features, and high-order statistical features, and optimizes features by combining feature importance scoring, correlation analysis, and principal component analysis to achieve multi-dimensional feature expression of node operation status. This method not only improves the expressive power of features, but also reduces data redundancy and computational complexity through dimensionality reduction processing;

[0081] (3) The present invention adopts a three-layer detection architecture consisting of local, regional, and global layers, and implements step-by-step analysis from a single node to the global level through a hierarchical anomaly detection mechanism. This architecture can effectively reduce data transmission overhead and achieve a reasonable distribution of computing load. At the same time, it improves the accuracy and scalability of anomaly detection through multi-level information fusion, providing efficient and reliable technical support for fault detection in large-scale distributed storage systems.

[0082] (4) The local layer uses an improved clustering algorithm, introduces local density calculation, time decay factor, and local entropy value, and realizes real-time clustering analysis of node status. This method adaptively selects cluster centers through a comprehensive scoring mechanism and uses dynamic thresholds for cluster division. This not only improves clustering accuracy, but also effectively adapts to the dynamic changes of node status, providing high-quality local analysis results for the regional layer.

[0083] (5) The regional layer merges clusters by calculating the inter-cluster similarity matrix and adaptive thresholds, and introduces the regional anomaly index (RAI) to comprehensively evaluate the density, isolation, and temporal correlation of clusters. This method effectively identifies and extracts abnormal patterns within the region. By generating a standardized regional state descriptor (RSD), it provides structured regional state information to the global layer, effectively reducing the amount of data transmission.

[0084] (6) The global layer constructs a regional association graph, implements spatiotemporal analysis of anomalies based on a sliding time window mechanism, and uses multi-dimensional feature fusion to obtain a global anomaly score (GAS). This method not only accurately identifies complex cross-regional faults but also predicts the propagation path and impact range of anomalies. By outputting a standardized anomaly detection result set (ADR), it provides comprehensive anomaly feature information for the fault diagnosis module. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0086] Figure 1 This is a system framework diagram of the present invention. DETAILED DESCRIPTION

[0087] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0088] like Figure 1 As shown, the present invention provides a distributed storage node fault detection system, comprising:

[0089] a data collection module configured to collect node operation status data and perform preprocessing and standardization on the node operation status data;

[0090] A feature extraction module is configured to perform feature calculation on the node operation status data and perform feature dimension reduction through principal component analysis to obtain a feature vector;

[0091] The distributed anomaly detection module adopts a three-layer detection architecture, including local layer, regional layer, and global layer. The local layer is configured to perform real-time cluster analysis on feature vectors using a clustering algorithm, the regional layer is configured to fuse the cluster analysis results of the local layer, and the global layer is configured to perform anomaly detection on the fused information.

[0092] A fault diagnosis module is configured to receive anomaly detection results at the global layer, classify faults, perform feature association analysis based on association rule mining, locate the root cause using a location algorithm, and generate a fault diagnosis report;

[0093] The early warning decision module is configured to classify confirmed faults based on multi-level early warning thresholds according to the fault diagnosis report, set the early warning suppression time window, match the response strategy through preset strategy templates and dynamically generated rules, and use a weighted scoring mechanism to prioritize the strategies, and select the preset strategy with the highest priority as the final recommended strategy.

[0094] Specifically, in one embodiment of the present invention, the data acquisition module, as a basic component of the distributed storage node fault detection system, is mainly responsible for collecting node operation status data and preprocessing and standardizing the collected data. The node operation status data collected by this module can be divided into three categories: system-level indicator data, storage-level indicator data, and network-level indicator data. Among them, system-level indicator data includes CPU usage, memory usage, system load, and process status; storage-level indicator data includes IO performance, disk usage, read and write bandwidth, and disk health status; network-level indicator data includes network latency, packet loss rate, bandwidth usage, and connection status. These indicator data comprehensively cover the operation status characteristics of the storage node, providing a reliable data foundation for subsequent anomaly detection and fault diagnosis.

[0095] After collecting node operation status data, the data acquisition module needs to preprocess and standardize the raw data to improve data quality and eliminate noise interference. The specific implementation process includes three steps: missing value processing, outlier smoothing, and data standardization. The process is as follows:

[0096] To address the data missing problem that occurs during the collection process, a multivariate linear regression interpolation method is used. This method fully considers the temporal and spatial correlation of the data and repairs the missing data by establishing a regression model. The specific implementation steps are as follows:

[0097] First, a multiple linear regression model is established based on historical data. For the missing indicator y, k indicators with strong correlation with it are selected as independent variables x1, x2, ..., x k , establish the regression equation:

[0098] y=β0+β1x1+β2x2+...+β k x k +ε

[0099] Among them, β0 is the intercept term, β1,β2,...,β k is the regression coefficient, and ε is the random error term.

[0100] Estimate the regression coefficients using the method of least squares:

[0101] β=(X T X) -1 X T Y

[0102] Among them, X is the independent variable matrix and Y is the dependent variable vector.

[0103] Substitute the current values ​​of the relevant indicators into the regression equation to obtain the predicted values ​​of the missing values:

[0104]

[0105] Combined with the confidence interval, the correction is made to ensure the rationality of the interpolation result. The confidence interval of the predicted value is defined as:

[0106]

[0107] where ∈ is the error bound calculated from the t-distribution:

[0108]

[0109] Among them, s is the regression standard deviation, t α / 2,n-k-1 is the critical value of the t distribution, which is usually found in the t distribution table and depends on the confidence level α and the degrees of freedom nk-1. x0 is the independent variable vector of the point to be predicted.

[0110] Adaptive median filtering is used to smooth outliers, and the window size is dynamically adjusted according to the degree of data fluctuation:

[0111] 1. Initialize the filter window size S min and the maximum window size S max :

[0112] S min : Initial window size, set to 3; S max : Maximum window size, determined by the data sampling frequency.

[0113] 2. Calculate the local volatility of the data in the current window:

[0114]

[0115] Where n is the number of data points in the window, x i is the data point in the window, is the mean of the data in the window.

[0116] 3. Dynamically adjust the window size according to the volatility:

[0117]

[0118] Among them, k is the adjustment coefficient, which is used to control the growth rate of the window size; Indicates a floor operation.

[0119] 4. Perform median filtering on the data in the current window: Calculate the median med of the data in the window; if the deviation between the current value and the median exceeds the preset threshold θ, replace it with the median: if |x-med|>θ, then x=med.

[0120] Use the Z-score method to standardize the data and introduce a skewness correction coefficient to correct the asymmetry of the data distribution. The specific implementation steps are as follows:

[0121] Calculate the skewness coefficient of the data:

[0122]

[0123] Where n is the number of samples, x i is the original data, is the sample mean.

[0124] Calculate the correction factor based on the skewness coefficient:

[0125] α=1 / (1+|SK|)

[0126] Perform Z-score normalization:

[0127] z=α×(x-μ) / σ

[0128] Where μ is the sample mean and σ is the sample standard deviation.

[0129] Verify the standardized results to ensure that: the mean of the standardized data is close to 0; the standard deviation is close to 1; and the skewness of the data distribution is effectively corrected.

[0130] Through the above three steps, the data acquisition module can effectively improve data quality and provide a reliable data foundation for subsequent feature extraction and anomaly detection.

[0131] Specifically, in one embodiment of the present invention, the feature extraction module is mainly responsible for performing feature calculation and optimization processing on the node operation status data. It adopts a multi-dimensional feature extraction method, calculates time domain features, frequency domain features, time-frequency features and high-order statistical features, and optimizes the extracted features. Finally, it performs dimensionality reduction through principal component analysis to obtain the final feature vector.

[0132] The time domain features include first-order moments, second-order moments, third-order moments and fourth-order moments. The frequency domain features obtain power spectral density and frequency band energy distribution through fast Fourier transform. The time-frequency features are decomposed at multiple scales based on the db4 wavelet basis. The high-order statistical features extract nonlinear features through the ARIMA model.

[0133] In this embodiment, the feature optimization process is as follows:

[0134] Calculate the feature importance score SI(f i ), sort the features according to their importance scores, select the top N features to be retained, and the feature importance score SI(f i ) is calculated as follows:

[0135] SI(f i )=α1·IG(f i )+α2·GR(fi )+α3·Relief(f i )

[0136] Where, f i represents the i-th feature, IG(f i ) is the information gain value, GR(f i ) is the information gain rate, Relief(f i ) is the Relief algorithm score, α1, α2, and α3 are weight coefficients and satisfy α1+α2+α3=1.

[0137] Specifically, the Relief algorithm is a feature selection algorithm used to evaluate the importance of features. It evaluates the importance of features by analyzing the ability of each feature to distinguish neighboring samples. The algorithm process is as follows:

[0138] For each feature f, initialize the weight W[f] = 0

[0139] Repeat m times:

[0140] Randomly select a sample R

[0141] Find the k nearest similar samples of R (called Nearest Hits, H for short)

[0142] Find the k nearest different class samples of R (called Nearest Misses, M for short)

[0143] For each feature f:

[0144] W[f]=W[f]-Σ(diff(f,R,H) / (m×k))+Σ(diff(f,R,M) / (m×k))

[0145] Among them, for numerical features, diff(f,R,H)=|value(f,R)-value(f,H)| / (max(f)-min(f)); for discrete features, diff(f,R,H)=0,if value(f,R)=value(f,H),diff(f,R,H)=1,ifvalue(f,R)≠value(f,H).

[0146] The correlation between features is calculated based on the Pearson correlation coefficient, and highly correlated redundant features are eliminated by setting the correlation threshold θ:

[0147]

[0148] In the formula, R(f i ,f j ) is the feature f i With feature fj The feature correlation, Cov(f i ,f j ) is the feature f i With feature f j The covariance of and They are respectively features f i With feature f j The standard deviation of .

[0149] Perform principal component analysis and dimensionality reduction on the retained features: calculate the feature covariance matrix, solve the eigenvalues ​​and eigenvectors, sort by eigenvalue size, select the main components, calculate the cumulative variance contribution rate, determine the feature dimension k after dimensionality reduction, and generate the eigenvector F after dimensionality reduction {f′1,f′2,...,f′ k}, where the criterion for determining the dimension k of the dimensionality reduction is: the cumulative variance contribution rate of the first k principal components is not less than a preset threshold η, where η is specifically set to 85% or 90%.

[0150] Specifically, in one embodiment of the present invention, a three-layer detection architecture is used to detect node anomalies. From a functional perspective, the local layer is responsible for real-time monitoring at the node level, the regional layer performs collaborative analysis within the region, and the global layer completes system-level anomaly detection. From the perspective of data granularity, the data processed by each layer is abstracted step by step: the local layer processes the original monitoring data, the regional layer processes the clustered feature data, and the global layer processes the regional status descriptor. This layered design provides good system scalability and supports horizontal expansion and independent optimization and upgrading of the system. At the same time, by dispersing computing pressure, a balanced distribution of computing load is achieved, which improves the system response speed. In addition, the multi-level cross-validation mechanism effectively improves detection accuracy and reduces the false alarm rate.

[0151] Specifically, the distributed anomaly detection module executes a layered, progressive process: the local layer performs real-time cluster analysis and generates results; the regional layer collects and integrates the clustering results of nodes within the region to calculate the regional anomaly index; and the global layer constructs a state matrix, performs spatiotemporal analysis, and ultimately generates anomaly detection results. Clear data flow between layers ensures efficient information transmission and processing.

[0152] First, the local layer, as the first layer of distributed anomaly detection, is primarily responsible for real-time anomaly detection of individual nodes, quickly identifying local anomaly patterns through cluster analysis. Its characteristics include: rapid response to node status changes; the use of lightweight algorithms that do not affect normal node operations; and the ability to dynamically adjust parameters based on data characteristics. The clustering algorithm execution process is as follows:

[0153] A1. Normalize the eigenvectors and map them to the interval [0,1]. Treat each eigenvector as a point and calculate the Euclidean distance d between the two points.ij ;

[0154] A2. Calculate the local density of each point:

[0155] ρ i,t =∑ j χ(d ij -d c )·τ(t)

[0156] Where, ρ i,t is the local density, χ is the indicator function, d c is the cutoff distance, τ(t)=e -λt is the time attenuation factor, λ is the attenuation coefficient, and t is the time interval; ij -d c <0, χ=1, otherwise χ=0; d c The value may be the 2% quantile of all point-to-point distances of the data point. In this embodiment, the value range of λ is 0.1-0.5.

[0157] A3. Calculate the minimum distance δ from each point to the high-density point i =min(d ij ), where j satisfies ρ j,t >ρ i,t ;

[0158] A4. Calculate the overall score:

[0159] γ i =ρ i,t ·ε i ·(1+ε·En(i))

[0160] Where, γ i is the comprehensive score, En(i) is the local entropy value of point i, and ε is the entropy weight coefficient; where En(i) = -∑(p j ×log(p j )), where p j is the proportion of points of type j in the neighborhood of point i. The local entropy value represents the uncertainty of data distribution in the local area. In this embodiment, the value range of ε is 0.1-0.3.

[0161] A5. Determine the local density threshold:

[0162] th den =μ·mean(ρ)+σ·std(ρ)

[0163] Where th den represents the local density threshold, μ is the mean adjustment coefficient, σ is the standard deviation adjustment coefficient, mean(ρ) is the mean value of the local density, and std(ρ) is the standard deviation of the local density;

[0164] Calculate the distance anomaly threshold:

[0165] th dis =κ1·d max +κ2·std(D)

[0166] Where th dis is the distance anomaly threshold, κ1 is the distance benchmark coefficient, κ2 is the fluctuation adjustment coefficient, d max is the maximum distance of the current cluster, std(D) is the standard deviation of the distance within the cluster;

[0167] In this embodiment, the value range of μ is 1.0-1.5, the value range of σ is 2.0-3.0, the value range of κ1 is 0.6-0.8, and the value range of κ2 is 1.5-2.0.

[0168] A6. Based on the comprehensive score γ i Determine the cluster center according to the local density threshold th den Perform cluster division and use the distance anomaly threshold th dis Identify outliers;

[0169] A7. Based on the cluster division and outlier identification results, generate local clustering results, including cluster information {c,n c ,d max} and the set of outliers, the cluster information includes the cluster center c, the number of vectors in the cluster n c and the maximum distance d max .

[0170] As the middle layer of distributed anomaly detection, the regional layer is primarily responsible for fusing and analyzing the clustering results of multiple nodes within the same region to identify regional anomaly patterns. Its features include: integrating detection results from multiple nodes; filtering local noise through result fusion; and supporting the dynamic addition and removal of nodes. The fusion processing flow is as follows:

[0171] B1, receiving the local clustering results of each node in the area, including the cluster center set C = {c1, c2, ..., c m}、N={n1,n2,...,n m}、Maximum distance set And the set of outliers A=a1,a2,...,a k ;

[0172] B2. Calculate the inter-cluster similarity matrix S, for any two cluster centers c x and c y , and its similarity is calculated as follows:

[0173]

[0174] Where, s(c x ,c y ) represents the cluster center c x and c y The similarity of , σ is the Gaussian kernel parameter, determined by cross-validation; ‖c x -c y ‖ represents the Euclidean distance between the two cluster centers, n x ,n y represents the number of vectors in the two clusters, Indicates the maximum distance between two clusters;

[0175] B3. Construct a hierarchical clustering tree T and use the adaptive threshold method to merge clusters:

[0176] Merge th =β1×(mean(S)+β2×std(S))

[0177] In the formula, Merge th is the merging threshold, β1 is the benchmark coefficient, β2 is the variance adjustment coefficient, mean(S) is the mean of the similarity matrix S, and std(S) is the standard deviation of the similarity matrix S. In this embodiment, the value range of β1 is 0.8-1.2, and the value range of β2 is 1.5-2.5.

[0178] When merging, it is iterated from bottom to top, each time the cluster pair with the highest similarity is selected for merging, and the characteristics of the merged cluster are updated: the cluster center and the maximum distance are recalculated.

[0179] B4. Calculate the regional anomaly index RAI:

[0180] RAI(c)=w1×DS(c)+w2×IS(c)+w3×TS(c)+w4×AD(c)

[0181] Where DS(c) is the density score, which indicates the density of the cluster; IS(c) is the isolation score, which indicates the degree of isolation of the cluster from other clusters; TS(c) is the temporal correlation score, which indicates the temporal evolution characteristics of the cluster; AD(c) is the anomaly distribution feature, which reflects the distribution characteristics of the anomalies around the cluster; w1, w2, and w3 are weight coefficients, and they satisfy w1+w2+w3=1.

[0182] In this embodiment, n(c) represents the number of points in the cluster, d max (c) is the maximum distance within the cluster; dis(c,c j ) represents cluster c and other clusters c j The center distance between them, min(dis(c,c j)) represents the distance from cluster c to the nearest cluster, mean(dis(c,all other-c )) represents the average distance from cluster c to all other clusters; TS(c) = e -λ*Δt *sta(c), where λ is the time decay coefficient, Δt is the time interval from the current moment to the first appearance of cluster c, and sta(c) is the stability measure of cluster c, which is calculated based on the changes in cluster c during the historical window period; AD(c) = count(r) / total(r), where r is the influence radius of cluster c, which is d max 1.5 times of, count(r) represents the number of outliers within the impact radius r, and total(r) is the total number of points within the impact radius r.

[0183] B5. Generate regional status descriptor RSD={C * ,N * ,D * ,RAI * ,T *}, where C * is the set of cluster centers after merging, N * is the updated set of vectors in the cluster, D * is the updated maximum distance set, RAI * is the set of abnormal indices, T * is the timestamp.

[0184] The global layer, as the highest level of distributed anomaly detection, is responsible for integrating detection results from all regions and performing system-level anomaly analysis. Its characteristics include the ability to discover cross-regional anomaly patterns, predict anomaly propagation trends, and provide a basis for system-level responses. The anomaly detection process is as follows:

[0185] C1. Construct the global state matrix GSM = [RSD1; RSD2; ...; RSD r ], where r is the total number of regions;

[0186] C2. Calculate the inter-region correlation strength matrix (RCM) and establish a regional correlation relationship diagram by analyzing the similarity of performance indicators, network topology distance, and the similarity of anomaly patterns:

[0187] RCM(a,b)=∈1×PS(a,b)+∈2×TS(a,b)+∈3×AS(a,b)

[0188] Where RCM(a,b) represents the correlation strength matrix between region a and region b, PS(a,b) is the performance index similarity between region a and region b, TS(a,b) is the topological distance similarity between region a and region b, AS(a,b) is the anomaly pattern similarity between region a and region b, ∈1, ∈2, ∈3 are weight coefficients, and ∈1+∈2+∈3=1;

[0189] In this embodiment, PS(a,b)=1-||P a -P b ‖ / max(‖P a ‖,‖P b ‖), P a 、P b is the performance index vector of regions a and b, ‖P a -P b ‖ is the Euclidean distance between two vectors, ‖P a ‖,‖P b ‖ is the norm of the two vectors. hop(a,b) is the minimum number of hops between areas a and b, max hop is the maximum number of hops in the network. AS(a,b)=Jaccard(A a ,A b )*SIM(Y a ,Y b ), A a ,A b is the set of outliers in regions a and b, Jaccard(A a ,A b ) is the Jaccard similarity of two outlier sets, Y a ,Y b is the abnormal pattern feature vector of regions a and b, SIM(Y a ,Y b ) is the cosine similarity of the abnormal pattern feature vector.

[0190] C3. Using a sliding time window mechanism, we analyze anomaly patterns in both temporal and spatial dimensions. This includes tracking the evolution of anomaly patterns, identifying the propagation paths of anomalies between regions, and predicting the likely direction of anomaly spread. Specifically, using the sliding time window mechanism, we perform a time series analysis of each region's anomaly index (RAI), calculating the rate of change and fluctuation characteristics. Using the regional correlation matrix (RCM), we analyze the diffusion paths of anomalies between different regions and construct a propagation map. Combining historical propagation paths with current anomaly trends, we predict the likely direction of anomaly spread and the scope of impact.

[0191] C4. Comprehensively evaluate the degree of abnormality by weightedly fusing local abnormality features, regional abnormality features, and associated abnormality features to obtain the global abnormality score GAS.

[0192] GAS=ω1*LAF+ω2*RAF+ω3*CAF

[0193] Among them, LAF is the local abnormal feature, LAF=∑(RAI*weight local ), weight local is the weight based on local density and centrality; RAF is the regional anomaly feature, RAF = En(RCM) * max(RAI), En(RCM) is the local entropy value of RCM; CAF is the associated anomaly feature, CAF = prop score *Im score , prop score is a score based on propagation speed and direction consistency, Im score = Number of affected nodes / total number of nodes.

[0194] C5. Adopt an adaptive threshold method to determine anomalies. The threshold is dynamically adjusted based on the statistical characteristics of historical anomaly data. Specifically, it includes:

[0195] Calculate the mean and standard deviation of historical GAS scores; consider time trends and seasonal factors; introduce an adaptive adjustment coefficient to dynamically adjust based on the current anomaly ratio; set minimum and maximum threshold boundaries to ensure that the threshold fluctuates within a reasonable range.

[0196] C6. Output the standardized anomaly detection result set ADR, which includes the global anomaly score GAS, the location information AL of the abnormal area, the quantitative index of the anomaly degree AQ, the propagation situation feature vector PF, the impact range prediction IR, and the timestamp T.

[0197] Specifically, in one embodiment of the present invention, the fault diagnosis module receives the anomaly detection result set ADR as input, and realizes accurate diagnosis and location of the fault through the collaborative processing of four sub-units: fault classification, feature association analysis, root cause location and diagnostic report generation.

[0198] The fault classification unit builds a multi-level classification model based on the XGBoost decision tree algorithm. It uses the global anomaly score (GAS), the anomaly region feature (AL), and the propagation feature vector (PF) as feature inputs. The model is trained on a historical fault sample library to predict the probability of fault types and output a fault type vector (FT) containing confidence levels. Specifically, the XGBoost decision tree algorithm is used to construct a three-layer classification model. The first layer distinguishes major fault categories (hardware, software, and network failures), the second layer refines fault subcategories (such as disk failures, memory failures, and network congestion), and the third layer identifies specific fault types. The output is a fault type probability vector (FT), which includes confidence scores for each fault type and provides a probability distribution of the fault type.

[0199] Based on the temporal evolution data of abnormal regions and the propagation relationships between regions, the feature association analysis unit uses the Apriori algorithm to mine association rules between abnormal patterns and fault types, construct a fault feature map G, and establish a feature association strength matrix FCM for each fault type. Specifically, the temporal variation characteristics of performance indicators in abnormal regions are extracted, the correlation and causal relationships between indicators are analyzed, and the variation patterns of key performance indicators are identified. The Apriori algorithm is used to mine association rules, setting minimum support and confidence thresholds, extracting high-frequency abnormal pattern combinations, and establishing a mapping relationship between abnormal patterns and fault types. A fault feature association graph G is constructed, where nodes represent performance indicators or abnormal events, edges represent associations between indicators, and edge weights reflect the strength of association. The fault feature association strength matrix FCM is constructed, and the direct and indirect association strengths are calculated.

[0200] The root cause location unit takes the area where the anomaly first occurs as the starting point, combines the propagation situation feature PF and network topology information, uses the random walk algorithm to analyze the fault propagation path, uses the PageRank algorithm to evaluate the importance of nodes, and outputs the root cause probability ranking list RC and the fault impact range assessment report IR;

[0201] The diagnostic report generation unit integrates the fault type FT, feature correlation map G, and root cause analysis results RC to generate a standardized diagnostic report DR that includes fault location, impact assessment, and repair suggestions, and provides visual fault diagnosis results. The report content may include:

[0202] Fault overview: including fault type, severity, and impact scope;

[0203] Root cause analysis: includes root cause location results and credibility assessment;

[0204] Propagation analysis: including fault propagation path and diffusion trend;

[0205] Impact assessment: including performance impact and potential risk analysis;

[0206] Repair suggestions: including treatment plans and expected results.

[0207] Specifically, in one embodiment of the present invention, the early warning decision module receives the anomaly detection result set ADR and the fault diagnosis report DR as input, and realizes intelligent decision-making of graded early warning and processing strategies for faults through the coordinated processing of four sub-units: fault level assessment, early warning control, strategy matching and decision optimization.

[0208] The fault level assessment unit comprehensively analyzes the global anomaly score GAS, fault type probability distribution FT and impact range prediction IR, establishes a multi-dimensional fault severity assessment system, and dynamically generates the fault level assessment result SL; specifically, based on the assessment results, the faults are divided into four levels: emergency level, severe level, warning level and prompt level.

[0209] The early warning control unit implements a time-window-based early warning triggering mechanism based on the fault level (SL) and propagation profile (PF). Alarm priority scores determine the order of alarms. For example, emergency level: no suppression time; severe level: 5-minute suppression window; warning level: 15-minute suppression window; and prompt level: 30-minute suppression window. Faults of the same type are merged within the time window, and faults of different levels are triggered according to the highest level. Root cause analysis of the associated faults is then performed and a unified early warning is issued.

[0210] The strategy matching unit selects a matching solution from the preset strategy library based on the current fault type FT and the root cause analysis result RC;

[0211] The decision optimization unit performs multi-objective optimization on candidate processing strategies, comprehensively considering factors such as fault severity SL, impact range IR, processing cost and risk, and outputs the optimal strategy execution plan and execution sequence.

[0212] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A distributed storage node fault detection system, characterized in that: include: a data collection module configured to collect node operation status data and perform preprocessing and standardization on the node operation status data; A feature extraction module is configured to perform feature calculation on the node operation status data and perform feature dimension reduction through principal component analysis to obtain a feature vector; The distributed anomaly detection module adopts a three-layer detection architecture, including a local layer, a regional layer, and a global layer. The local layer is configured to perform real-time cluster analysis on feature vectors through a clustering algorithm based on local density and time decay factor, calculate the local density of each point, the minimum distance to the high-density point, and the comprehensive score, divide the clusters according to the local density threshold, and use the distance anomaly threshold to identify anomalies; the regional layer is configured to calculate the inter-cluster similarity matrix and merge the clusters using an adaptive threshold method, calculate the regional anomaly index RAI, and comprehensively evaluate the density score, isolation score, time correlation score, and anomaly distribution characteristics of the cluster to generate a regional state descriptor; the global layer is configured to construct a global state matrix, calculate the inter-regional correlation strength matrix, perform spatiotemporal analysis on the anomaly pattern based on the sliding time window mechanism, weightedly fuse the local anomaly features, regional anomaly features, and associated anomaly features to obtain a global anomaly score, and use the adaptive threshold method to determine anomalies; A fault diagnosis module is configured to receive anomaly detection results at the global layer, classify faults, perform feature association analysis based on association rule mining, locate the root cause using a location algorithm, and generate a fault diagnosis report; The early warning decision module is configured to classify confirmed faults based on multi-level early warning thresholds according to the fault diagnosis report, set the early warning suppression time window, match the response strategy through preset strategy templates and dynamically generated rules, and use a weighted scoring mechanism to prioritize the strategies, and select the preset strategy with the highest priority as the final recommended strategy.

2. A distributed storage node fault detection system according to claim 1, characterized in that: Node operation status data is divided into three categories according to the monitoring indicator type: system-level indicator data, storage-level indicator data, and network-level indicator data. System-level metrics include CPU usage, memory usage, system load, and process status; Storage-level metrics include I / O performance, disk usage, read / write bandwidth, and disk health status. Network-level metrics include network latency, packet loss rate, bandwidth usage, and connection status.

3. A distributed storage node fault detection system according to claim 2, characterized in that: The data acquisition module performs preprocessing and standardization on the node operation status data, including the following steps: The collected original data were processed for missing values, and the multivariate linear regression interpolation method was used to repair the data by combining temporal correlation and spatial correlation; Adaptive median filtering is used to smooth outliers, and the window size is dynamically adjusted according to the degree of data fluctuation; The Z-score method was used to standardize the data, and the skewness correction coefficient was introduced to correct the asymmetry of the data distribution.

4. A distributed storage node fault detection system according to claim 2, characterized in that: When the feature extraction module calculates the features of the node operation status data, it uses a multi-dimensional feature extraction method to calculate the time domain features, frequency domain features, time-frequency features and high-order statistical features, and optimizes the extracted features; The time domain features include first-order moments, second-order moments, third-order moments and fourth-order moments. The frequency domain features obtain power spectral density and frequency band energy distribution through fast Fourier transform. The time-frequency features are decomposed at multiple scales based on the db4 wavelet basis. The high-order statistical features extract nonlinear features through the ARIMA model.

5. A distributed storage node fault detection system according to claim 4, characterized in that: The optimization processing of the extracted features includes: Calculate the feature importance score SI(f i ), sort the features according to their importance scores, select the top N features to be retained, and the feature importance score SI(f i ) is calculated as follows: SI(f i )=α1·IG(f i )+α2·GR(f i )+α3·Relief(f i ) Where, f i represents the i-th feature, IG(f i ) is the information gain value, GR(f i ) is the information gain rate, Relief(f i ) is the score of the Relief algorithm, α1, α2, and α3 are weight coefficients and satisfy α1+α2+α3=1; The correlation between features is calculated based on the Pearson correlation coefficient, and highly correlated redundant features are eliminated by setting the correlation threshold θ: In the formula, R(f i , f j ) is the feature f i With feature f j The feature correlation, Cov(f i , f j ) is the feature f i With feature f j The covariance of and They are respectively features f i With feature f j The standard deviation of The retained features are subjected to principal component analysis and dimensionality reduction processing to obtain the reduced dimensionality feature vector F = {f′1, f′2, ..., f′ k }, where k is the feature dimension after dimensionality reduction, and the k value is determined by calculating the cumulative variance contribution rate, so that the cumulative variance contribution rate of the first k principal components is not less than the preset threshold η.

6. A distributed storage node fault detection system according to claim 2, characterized in that: In the distributed anomaly detection module, the execution process of the clustering algorithm is as follows: A1. Normalize the eigenvectors and map them to the interval [0,1]. Treat each eigenvector as a point and calculate the Euclidean distance d between the two points. ij ; A2. Calculate the local density of each point: r i，t =∑ j x(d ij -d c )·τ(t) Where, ρ i，t is the local density, χ is the indicator function, d c is the cutoff distance, τ(t)=e -λt is the time attenuation factor, λ is the attenuation coefficient, and t is the time interval; A3. Calculate the minimum distance δ from each point to the high-density point i =min(d ij ), where j satisfies ρ j,t >ρ i,t ; A4. Calculate the overall score: γ i =ρ i,t ·δ i ·(1+ε·En(i)) Where, γ i is the comprehensive score value, En(i) is the local entropy value of point i, and ε is the entropy weight coefficient; A5. Determine the local density threshold: th den =μ·mean(ρ)+σ·std(ρ) Where th den represents the local density threshold, μ is the mean adjustment coefficient, σ is the standard deviation adjustment coefficient, mean(ρ) is the mean value of the local density, and std(ρ) is the standard deviation of the local density; Calculate the distance anomaly threshold: th dis =κ1·d max +κ2·std(D) Where th dis is the distance anomaly threshold, κ1 is the distance benchmark coefficient, κ2 is the fluctuation adjustment coefficient, d max is the maximum distance of the current cluster, std(D) is the standard deviation of the distance within the cluster; A6. Based on the comprehensive score γ i Determine the cluster center according to the local density threshold th den Perform cluster division and use the distance anomaly threshold th dis Identify outliers; A7. Based on the cluster division and outlier identification results, generate local clustering results, including cluster information {c,n c ,d max } and the set of outliers, the cluster information includes the cluster center c, the number of vectors in the cluster n c and the maximum distance d max .

7. A distributed storage node fault detection system according to claim 6, characterized in that: The fusion processing of the cluster analysis results of the local layer includes: B1, receiving the local clustering results of each node in the area, including the cluster center set C = {c1, c2, ..., c m }、N={n1,n2,...,n m }、Maximum distance set And the set of outliers A=a1,a2,...,a k ; B2. Calculate the inter-cluster similarity matrix S, for any two cluster centers c x and c y , and its similarity is calculated as follows: Where, s(c x , c y ) represents the cluster center c x and c y Similarity, σ is the Gaussian kernel parameter, determined by cross-validation; ||c x -c y || represents the Euclidean distance between the centers of two clusters, n x , n y represents the number of vectors in the two clusters, Indicates the maximum distance between two clusters; B3. Construct a hierarchical clustering tree T and use the adaptive threshold method to merge clusters: Merge th =β1×(mean(S)+β2×std(S)) In the formula, Merge th is the merging threshold, β1 is the benchmark coefficient, β2 is the variance adjustment coefficient, mean(S) is the mean of the similarity matrix S, and std(S) is the standard deviation of the similarity matrix S; B4. Calculate the regional anomaly index RAI: RAI(c)=w1×DS(c)+w2×IS(c)+w3×TS(c)+w4×AD(c) Where DS(c) is the density score, which indicates the density of the cluster; IS(c) is the isolation score, which indicates the degree of isolation of the cluster from other clusters; TS(c) is the temporal correlation score, which indicates the temporal evolution characteristics of the cluster; AD(c) is the anomaly distribution feature, which reflects the distribution characteristics of the anomalies around the cluster; w1, w2, and w3 are weight coefficients, and they satisfy w1+w2+w3=1. B5. Generate regional status descriptor RSD={C * , N * , D * , RAI * , T * }, where C * is the set of cluster centers after merging, N * is the updated set of vectors in the cluster, D * is the updated maximum distance set, RAI * is the set of abnormal indices, T * is the timestamp.

8. A distributed storage node fault detection system according to claim 7, characterized in that: The process of performing anomaly detection at the global layer includes: C1. Construct the global state matrix GSM = [RSD1; RSD2; ...; RSD r ], where r is the total number of regions; C2. Calculate the inter-region correlation strength matrix (RCM) and establish a regional correlation relationship diagram by analyzing the similarity of performance indicators, network topology distance, and the similarity of anomaly patterns: RCM(a,b)=∈1×PS(a,b)+∈2×TS(a,b)+∈3×AS(a,b) Where RCM(a,b) represents the correlation strength matrix between region a and region b, PS(a,b) is the performance index similarity between region a and region b, TS(a,b) is the topological distance similarity between region a and region b, AS(a,b) is the anomaly pattern similarity between region a and region b, ∈1, ∈2, ∈3 are weight coefficients, and ∈1+∈2+∈3=1; C3. Based on the sliding time window mechanism, analyze the anomaly pattern in the spatiotemporal dimension, including tracking the evolution trend of the anomaly state, identifying the propagation path of the anomaly between regions, and predicting the possible spread direction of the anomaly; C4. Comprehensively evaluate the degree of abnormality by weightedly fusing local abnormality features, regional abnormality features, and associated abnormality features to obtain the global abnormality score GAS. C5. Adopting an adaptive threshold method to determine anomalies. The threshold is dynamically adjusted based on the statistical characteristics of historical anomaly data. C6. Output the standardized anomaly detection result set ADR, which includes the global anomaly score GAS, the location information AL of the abnormal area, the quantitative index of the anomaly degree AQ, the propagation situation feature vector PF, the impact range prediction IR, and the timestamp T.

9. A distributed storage node fault detection system according to claim 8, characterized in that: The fault diagnosis module receives the anomaly detection result set ADR as input and processes it through the following subunits: The fault classification unit builds a multi-level classification model based on the XGBoost decision tree algorithm. It uses the global anomaly score GAS, the abnormal area feature AL, and the propagation feature vector PF as feature inputs. It trains the model based on the historical fault sample library to achieve probabilistic prediction of the fault type and outputs the fault type vector FT containing the confidence level. The feature association analysis unit uses the Apriori algorithm to mine the association rules between abnormal patterns and fault types based on the time series evolution data of abnormal areas and the propagation relationship between regions, constructs the fault feature map G, and establishes the feature association strength matrix FCM for each fault type; The root cause location unit takes the area where the anomaly first occurs as the starting point, combines the propagation situation feature PF and network topology information, uses the random walk algorithm to analyze the fault propagation path, uses the PageRank algorithm to evaluate the importance of nodes, and outputs the root cause probability ranking list RC and the fault impact range assessment report IR; The diagnostic report generation unit integrates the fault type FT, feature correlation map G and root cause analysis results RC to generate a standardized diagnostic report DR that includes fault location, impact assessment and repair suggestions, and provides visual fault diagnosis results.

10. A distributed storage node fault detection system according to claim 9, characterized in that: The early warning decision module processes the anomaly detection result set ADR and the fault diagnosis report DR through the following subunits: The fault level assessment unit comprehensively analyzes the global anomaly score GAS, fault type probability distribution FT, and impact range prediction IR, establishes a multi-dimensional fault severity assessment system, and dynamically generates the fault level assessment result SL; The early warning control unit implements a time window-based early warning trigger mechanism based on the fault level SL and the propagation situation feature PF, and determines the alarm sequence through the early warning priority score; The strategy matching unit selects a matching solution from the preset strategy library based on the current fault type FT and the root cause analysis result RC; The decision optimization unit performs multi-objective optimization on candidate processing strategies, comprehensively considering factors such as fault severity SL, impact range IR, processing cost and risk, and outputs the optimal strategy execution plan and execution sequence.

Images