An adaptive operation and maintenance root cause location method and system based on deep learning

Abstract

The present invention discloses a method and system for adaptive operation and maintenance root cause location based on deep learning. The method first collects multi-source data such as system logs, performance indicators, alarm events and configuration changes, extracts time series features, text features and correlation features; combines the 3-sigma criterion and autoencoder to perform anomaly detection; when an anomaly is found, uses the fault knowledge graph to perform reasoning diagnosis and combines it with the XGBoost classification algorithm to achieve fault diagnosis; then performs fault propagation analysis and root cause location, and continuously optimizes the fault handling process. The present invention solves the problems of incomplete fault features, low detection accuracy, and limited diagnostic capabilities in existing operation and maintenance systems. By combining multi-source data fusion, deep learning detection, and knowledge graph reasoning, it achieves more accurate fault detection and root cause location, while having adaptive optimization capabilities, significantly improving the efficiency of operation and maintenance fault handling.

Description

Technical Field

[0001] The present invention relates to the field of computer system operation and maintenance technology, and in particular to a method and system for adaptive operation and maintenance root cause location based on deep learning. Background Art

[0002] With the rapid development of next-generation information technologies such as cloud computing, big data, and artificial intelligence, the scale and complexity of enterprise IT infrastructure and application systems continue to grow. The dependencies between components in large-scale distributed systems are complex, and a minor failure can trigger a chain reaction through inter-system dependencies, leading to even wider service anomalies. In this context, quickly and accurately locating the root cause of failures has become a key challenge in ensuring stable system operations.

[0003] Traditional O&M troubleshooting relies heavily on the experience of O&M personnel for analysis and diagnosis. O&M personnel manually collect system logs, monitoring data, and other information, and use their experience to infer the possible causes of the fault. This approach is adequate for simple faults, but it suffers from the following significant shortcomings when dealing with modern, complex IT systems: First, manual analysis is inefficient. Faced with massive amounts of system data and complex dependencies, manual analysis often takes a significant amount of time. Second, empirical judgments are highly subjective, and different O&M personnel may produce significantly different analysis results for the same fault. Third, experience is difficult to pass on, and the valuable experience accumulated by outstanding O&M personnel is difficult to effectively accumulate and reuse.

[0004] In recent years, with the development of machine learning technology, the industry has begun to experiment with applying artificial intelligence to fault diagnosis. Some approaches use text mining in system logs to identify abnormal patterns; others utilize time series analysis to detect abnormal performance indicators; and still others attempt to use deep learning models to learn normal system behavior patterns. While these approaches have improved the automation level of fault detection to some extent, they still have many limitations.

[0005] First, existing methods are often limited to a single data source when it comes to data usage. For example, analysis based solely on system logs can overlook fault information contained in other important data sources, such as performance metrics and alarms. Furthermore, methods that rely solely on performance metrics struggle to understand the specific context of the fault. This limitation of a single data source results in incomplete information about fault characteristics, impacting diagnostic accuracy.

[0006] Secondly, in terms of anomaly detection mechanisms, most methods use relatively simple detection algorithms. Statistical thresholding methods are susceptible to data noise, while single machine learning models struggle to cope with complex and changing failure modes. These simple detection mechanisms often perform poorly, generating numerous false positives and false negatives, especially when faced with new types of failures or multiple concurrent failures.

[0007] Third, in terms of knowledge management, existing systems generally lack effective mechanisms for managing and leveraging historical failure experience. Each time a failure occurs, a new analysis is required, failing to fully leverage historical experience in handling similar failures. This not only wastes resources but also prolongs troubleshooting time.

[0008] Fourth, when it comes to root cause location, many methods only focus on fault detection—they can detect system anomalies but struggle to pinpoint the specific root cause. This lack of in-depth analysis of the dependencies between system components makes it difficult to accurately determine the fault's propagation path, often requiring extensive troubleshooting.

[0009] Fifth, in terms of system adaptability, existing methods generally lack the ability to self-optimize and evolve. As new services are launched and system architectures evolve, failure modes are constantly changing. If the diagnostic system cannot adapt to these changes in a timely manner, its diagnostic effectiveness will gradually decline.

[0010] Therefore, to address these challenges, a new approach to root cause location is urgently needed. This approach should be able to integrate multi-source heterogeneous data to obtain comprehensive fault characteristics, employ advanced deep learning techniques to improve the accuracy of anomaly detection, leverage knowledge graphs to accumulate and reuse fault handling experience, achieve precise root cause location through systematic propagation analysis, and possess the capabilities of continuous learning and adaptive optimization. Summary of the Invention

[0011] The purpose of this invention is to propose an adaptive operation and maintenance root cause location method and system based on deep learning to address the problems of single data source, low diagnostic accuracy, and lack of adaptive optimization in existing operation and maintenance fault diagnosis, thereby improving the accuracy and efficiency of operation and maintenance fault root cause location. To achieve the above objectives, the technical solutions adopted by this invention are as follows:

[0012] An adaptive operation and maintenance root cause location method based on deep learning of abnormal features and graph reasoning includes the following steps:

[0013] S1 collects multi-source data, including system log data, performance indicator data, alarm event data, and configuration change data;

[0014] S2 performs data cleaning and standardization on the multi-source data, and establishes time series associations between the data;

[0015] S3 performs feature extraction on the preprocessed data to obtain time series features, text features, and association features;

[0016] S4 uses the 3-sigma criterion to perform single-indicator anomaly detection on the statistical indicators in the time series features. At the same time, it constructs the time series features, text features, and association features into a joint feature vector and uses an autoencoder to perform multi-indicator association anomaly detection and output an anomaly feature vector.

[0017] When S5 detects an anomaly, it performs reasoning diagnosis based on the fault knowledge graph and combines it with the XGBoost classification algorithm to perform fault diagnosis;

[0018] S6 performs fault propagation analysis and root cause location on the system based on the fault diagnosis results;

[0019] S7 updates the fault handling knowledge base based on the root cause location result and generates an optimized standard fault handling process.

[0020] Furthermore, S1 collects multi-source data including: collecting system log data through Agent; collecting performance indicator data through SNMP protocol and application monitoring interface; collecting alarm event data through SNMP Trap and timed polling; collecting configuration change data by monitoring configuration file changes and system parameter adjustment records.

[0021] Furthermore, the preprocessing step in S2 includes: performing data cleaning on the multi-source data, removing abnormal data and unifying the time format; performing data standardization on the cleaned multi-source data, unifying the format and measurement unit; and establishing a temporal association between performance indicator data, system log data, alarm event data and configuration change data within the same time window based on timestamps.

[0022] Furthermore, feature extraction in S3 includes:

[0023] S301. Use sliding window technology to extract the time series characteristics of performance indicator data, including: statistical characteristics of mean μ, standard deviation σ, rate of change r, as well as trend characteristics of rising / falling trend t, periodic fluctuation p, and mutation point d;

[0024] S302. The Drain algorithm is used to achieve rapid template extraction from large-scale log data. The Drain algorithm clusters similar log messages through a fixed-depth parse tree to identify log templates and calculates the feature vector V using the TF-IDF (term frequency-inverse document frequency) algorithm, where V contains the log keyword weight w, error code distribution e, and operation type frequency f;

[0025] S303. Construct the correlation feature matrix R within the time window T, where the matrix element R[i, j] represents the temporal correlation between the i-th alarm event and the j-th configuration change. The correlation is calculated by the alarm event time t a [i] and configuration change time t c The time difference of [j] is calculated.

[0026] Furthermore, anomaly detection in S4 includes:

[0027] S401. Perform 3-sigma statistical anomaly detection on the statistical characteristics of a single performance indicator, that is, detect the statistical indicators (mean μ, standard deviation σ, rate of change r) in the time series characteristics to identify significant deviations of a single indicator:

[0028] When |μ-μ base |>3σ base When , the mean is judged to be abnormal;

[0029] When |rr base |>3σ r When , the change rate is judged to be abnormal;

[0030] When|tt base |>3σ t When , the trend is judged to be abnormal;

[0031] where μ base 、 base 、r base , t base are the historical baseline means of the corresponding indicators, obtained through statistical analysis of historical normal data, σ r , σ t is the historical standard deviation of the corresponding indicator, which is used to measure the fluctuation range of the indicator and output the statistical detection result vector ST = [s a , s b , s c ],s a 、s b 、s c Respectively represent the abnormality degree of each statistical indicator;

[0032] S402. Construct a joint feature vector F = [F t , F l , F r ], where the time series feature vector is F t =[μ,σ,r,t,p,d], the text feature sub-vector is F l =[w, e, f], the associated eigenvector is F r =[R[i, j]];

[0033] S403: Combine the joint feature vector F=[F t , F l , F r ] Input autoencoder for anomaly detection:

[0034] The input feature F is compressed into a latent vector Z = Encoder(F) through the encoder, where Encoder is a dimensionality reduction encoding network that compresses high-dimensional features into a low-dimensional latent space;

[0035] Reconstruct the feature F′=Decoder(Z) through the decoder, where Decoder is a reconstruction decoding network that reconstructs the latent vector into the original feature space;

[0036] Calculate the root mean square error err between the input feature F and the reconstructed feature F'. This error reflects the abnormality of the feature. The formula is:

[0037]

[0038] Where i is the feature dimension index. When the reconstruction error err exceeds the preset threshold θ, it is judged as abnormal. The threshold θ is determined based on the reconstruction error distribution of normal samples, usually taking the 95% quantile of the reconstruction error distribution;

[0039] When an anomaly is detected, construct an abnormal feature vector AF = [Z, ST, δ F ], where Z is the potential vector output by the encoder, representing the main mode of the data, ST = [s a , s b , s c ] is the 3-sigma statistical test result, δ F =FF′ is the difference vector between the original feature and the reconstructed feature, which is used to locate abnormal features.

[0040] Furthermore, the fault knowledge graph in S5 includes:

[0041] The feature library is used to store typical fault features, including the abnormal pattern feature set H of performance indicators, where H includes typical abnormal features of CPU surge pattern, memory leak pattern, and network delay pattern;

[0042] Key log sequence feature set K, where K includes error log patterns, alarm log patterns, and abnormal operation sequence features;

[0043] A set of multi-indicator abnormal association patterns B, where B describes abnormal association rules between multiple performance indicators;

[0044] Case library, used to record actual fault instances, including the time when the fault occurred t s 、Affected range a r , Fault Level sl ;

[0045] The system environment information set Env at the time of the fault, including system configuration, operating status, load conditions, etc.

[0046] The associated mapping set U with the feature library, that is, U[i] indicates which features in the feature library the current fault case has;

[0047] The solution library is used to store fault handling methods, including the standard operating procedure sequence Q for fault handling, the execution condition set X for the handling steps, and the expected result set N;

[0048] A set U of mapping relationships with the case library, where U[i, j] represents the degree to which solution j is applicable to case i, and each mapping has a historical success rate p(y, s), where y represents the failure case, s represents the corresponding solution, and represents the historical success rate of solution s successfully solving failure case y, and an association with the feature library is established through the transfer relationship matrix C=U×U, and its association strength is calculated by w(h, s)×p(y, s), where w(h, s) represents the association weight between feature pattern h and solution s, which is calculated by the number of successful times of the solution corresponding to the feature pattern in historical cases.

[0049] Furthermore, the fault diagnosis in S5 includes:

[0050] S501. Knowledge graph reasoning based on graph traversal algorithm:

[0051] For the abnormal feature vector AF = [Z, ST, δ F ], calculate and each feature pattern h in the feature library i The similarity formula is:

[0052]

[0053] Among them, h i is the feature vector of the i-th fault mode in the feature library, |AF| and |h i |represents the Euclidean norm of the vector respectively, and the most similar known fault mode is found by calculating the similarity;

[0054] The feature-to-case association strength matrix M is established through the case mapping matrix U. The formula is:

[0055] M[i, j]=U[j][i]×g(h i ), i∈H, j∈Y

[0056] Among them, U[j][i] indicates whether the j-th fault case contains the i-th feature (0 or 1), g(h i ) is the characteristic h iThe importance coefficient in fault diagnosis is determined based on the success rate of the feature in historical diagnosis;

[0057] S502. Use graph traversal algorithm to calculate the current anomaly F to the historical case y j The accessibility score is:

[0058] score(y j )=∑(sim(AF,h i )×M[i,j]×I(h i )),y j ∈Y

[0059] Where I(h i ) represents the feature h i Impact factor, sim(AF, h i )) represents the similarity between the current abnormal feature and the historical feature pattern, M[i, j] represents the feature h i With case y j The correlation strength, I(h i ) represents the impact factor of feature hi, which is determined based on the position of the feature in the fault propagation link;

[0060] S503. Using XGBoost algorithm for machine learning classification:

[0061] The aforementioned joint feature vector AF = [Z, ST, δ F ] is input into the XGBoost classifier for training. The XGBoost classifier constructs a decision tree for each feature dimension, uses second-order Taylor expansion to optimize the loss function, adopts column sampling to reduce overfitting, and outputs the probability distribution P(c|F) of each fault type for subsequent diagnosis result fusion;

[0062] S504. Perform diagnostic result fusion for each fault type c:

[0063] Calculate the confidence of knowledge graph reasoning, the formula is:

[0064] conf kg (c) = sim(AF, h c )×case_support(c)

[0065] Among them, sim(AF, h c ) is the feature similarity between the current feature and the fault type c, case_support(c) is the historical case support;

[0066] Calculate the confidence of machine learning classification, the formula is:

[0067] conf ml(c)=P(c|F)×V(c)

[0068] Where P(c|F) is the probability output by the XGBoost classifier, and V(c) is the historical accuracy of the fault type;

[0069] S505. Determine the final diagnosis result according to the fusion rules:

[0070] When the knowledge graph reasoning and machine learning classification results are consistent, the consistent fault type is used to retrieve the corresponding solution from the knowledge graph solution library;

[0071] When the results are inconsistent, keep both diagnostic results and construct a diagnostic report, recording the diagnostic source, confidence level, reasoning basis, and recommended solutions for each result;

[0072] Furthermore, root cause location in S6 includes:

[0073] Based on the system environment information Env, a dependency directed graph G(N, E) is constructed, where the node set N represents the system components, the edge set E represents the dependency relationship between components, and the edge weight w dep (e i ) represents the dependency strength, where e i Represents the i-th edge in the dependency graph G(N, E);

[0074] For node n that detects an anomaly a , calculate the fault propagation probability, the formula is:

[0075] Prop(n j |n a )=Πw dep (e i ), e i ∈path(n a →n j )

[0076] Among them, path(n a →n j ) indicates that from n a to n j All edges on the path, w dep (e i ) represents the dependency edge e i The weight reflects the dependency strength between components and is determined by the call frequency between components (normalized to 0-1) and the degree of influence of the dependency relationship (0-1 score);

[0077] Construct a fault propagation tree T, which includes all nodes whose fault propagation probability exceeds the threshold δ:

[0078] T=n|Prop(n|n a)>δ,n∈N

[0079] Where δ is the preset propagation probability threshold (usually 0.1-0.3), which is used to filter out branches with low propagation probability and retain the main fault propagation path;

[0080] S602 assesses the scope of the fault impact:

[0081] Determine the set of directly affected components D, which includes components directly connected to the fault source node;

[0082] Determine the set of indirectly affected components I, which includes other affected components in the fault propagation tree;

[0083] Calculate the impact for each affected component v:

[0084] impact(v)=s(v)×u(v)×Prop(v|n a )

[0085] Where s(v) is the business importance score of component v (1-5), and u(v) is the percentage of affected users;

[0086] S603 prioritizes faults based on impact analysis, classifies faults according to their impact scope, business importance, and processing complexity, determines processing priorities based on the classification results, and updates the analysis results to the case library of the fault knowledge graph.

[0087] Furthermore, the optimization of the standard fault handling process in S7 includes:

[0088] S701 Establish a standard process library to store processing procedures:

[0089] The processing flow is represented as a directed acyclic graph W(Z, C), where the node set Z represents the operation steps and the edge set C represents the execution order between the steps;

[0090] Construct an execution condition vector for each process, recording the prerequisites required to execute the process, including system environment requirements and resource conditions;

[0091] Build a set of expected results, record the expected troubleshooting results of each process, and only retain results with a high historical success rate;

[0092] S702 extracts a valid operation sequence from the historical records:

[0093] Score each operation step: execution success rate (0-100%), average execution time (minutes), resource consumption level (1-5);

[0094] Based on the comprehensive evaluation of the above three dimensions, the steps are marked as "recommended", "optional" and "avoid";

[0095] S703 Identify key decision points:

[0096] Mark steps that meet any of the following conditions as key decision points: steps with multiple optional branches, checkpoints requiring manual judgment, and configuration steps that affect subsequent operations;

[0097] S704 update process library:

[0098] Evaluate new processing cases: whether they solve the problem, whether they are more efficient than the existing process, and whether they provide new processing ideas;

[0099] As new cases prove to be more effective, they are added to the process library.

[0100] The present invention also provides a system for adaptive operation and maintenance root cause location based on deep learning of abnormal features and graph reasoning, which is characterized by including:

[0101] The data acquisition and preprocessing module is used to collect and preprocess multi-source data and pass the preprocessed data to the feature extraction module;

[0102] A feature extraction module is used to receive the pre-processed data to extract features and transmit the extracted features to the anomaly detection module;

[0103] an anomaly detection module, configured to receive the features and perform anomaly detection, and trigger the fault diagnosis module when an anomaly is detected;

[0104] A fault diagnosis module is used to diagnose the detected anomaly and output the diagnosis results to the fault propagation analysis module;

[0105] a fault propagation analysis module, configured to receive the diagnosis results, perform propagation analysis and root cause location, and transmit the analysis results to the process optimization module;

[0106] A process optimization module is used to establish and optimize a standard fault handling process based on the analysis results. The optimized process is stored and used to guide subsequent fault handling.

[0107] The beneficial effects of the present invention are as follows:

[0108] (1) This invention innovatively combines 3-sigma statistical detection with deep learning detection based on autoencoders. The 3-sigma criterion is used to identify significant deviations from a single indicator, while the autoencoder discovers abnormal correlations between multiple indicators through feature reconstruction. This dual detection mechanism maintains the interpretability of statistical methods while utilizing deep learning to improve the ability to identify complex abnormal patterns, effectively reducing the false alarm rate.

[0109] (2) This paper constructs a fault knowledge graph containing a feature library, a case library, and a solution library. It achieves accurate diagnosis by fusing the results of graph reasoning and the XGBoost classification algorithm. The knowledge graph accumulates historical experience, while machine learning provides adaptive capabilities. The combination of the two ensures both diagnostic accuracy and the ability to handle new faults.

[0110] (3) This invention achieves precise root cause identification by constructing a system component dependency graph and analyzing the fault propagation path and impact range. Compared to the traditional problem-by-problem troubleshooting method, this graph-based analysis method can quickly identify the fault source and assess its impact, significantly improving fault handling efficiency.

[0111] (4) This invention establishes a standard fault handling process library and achieves adaptive evolution of the process through continuous evaluation and optimization. The system continuously improves the processing process by extracting effective operation sequences and identifying key decision points, making fault handling more standardized and efficient while reducing dependence on human experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0112] In order to more clearly illustrate the implementation of the present invention or the existing technical solutions, the following briefly introduces the drawings required for use in the embodiments or the description of the existing technologies.

[0113] Figure 1 This is the architecture diagram of the adaptive operation and maintenance root cause positioning system.

[0114] Figure 2 This is the structure diagram of the autoencoder network.

[0115] Figure 3 This is the fault knowledge graph structure diagram.

[0116] Figure 4 Propagate dependency graph for failures. DETAILED DESCRIPTION

[0117] The following is a clear and complete description of the technical solutions in the embodiments of the present invention, in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts are within the scope of protection of the present invention.

[0118] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0119] like Figure 1 As shown, the main steps of the method provided by the present invention are as follows:

[0120] (1) Collecting multi-source data, including system log data, performance indicator data, alarm event data, and configuration change data;

[0121] (2) performing data cleaning and standardization on the multi-source data, and establishing temporal associations between the data;

[0122] (3) Extract features from the preprocessed data to obtain temporal features, text features, and association features;

[0123] (4) The 3-sigma criterion is used to perform single-indicator anomaly detection on the statistical indicators in the time series features. At the same time, the time series features, text features and association features are constructed into a joint feature vector and an autoencoder is used to perform multi-indicator association anomaly detection and output the anomaly feature vector;

[0124] (5) When an anomaly is detected, reasoning diagnosis is performed based on the fault knowledge graph and combined with the XGBoost classification algorithm for fault diagnosis;

[0125] (6) Analyze the fault propagation and locate the root cause of the system based on the fault diagnosis results;

[0126] (7) Based on the root cause location results, the fault handling knowledge base is updated and an optimized standard fault handling process is generated.

[0127] As a preferred embodiment of the present invention, in step (1), the process of collecting multi-source data includes: (1-1) collecting system log data through Agent: deploying a lightweight Agent program to collect system operation logs in real time through file reading, system call tracking, etc.;

[0128] (1-2) Collect performance indicator data through the SNMP protocol and application monitoring interface: including system-level performance indicators such as CPU utilization, memory usage, network traffic, disk I / O, and application-level performance indicators such as application response time, number of connections, and queue length;

[0129] (1-3) Collect alarm event data through SNMP Trap and scheduled polling: Receive SNMP Trap alarms actively reported by devices and perform scheduled polling to check the alarm status;

[0130] (1-4) Collect configuration change data by monitoring configuration file changes and system parameter adjustment records: including configuration file modifications, system parameter adjustments, software version upgrades and other change information.

[0131] As a preferred embodiment of the present invention, in step (2), data preprocessing includes:

[0132] (2-1) Data cleaning: remove duplicate data, outliers and noise data, and unify data formats and time standards;

[0133] (2-2) Data standardization: standardize the format and unit conversion of data from different sources to achieve data standardization;

[0134] (2-3) Time series association: Establish associations between multi-source data within the same time window to form a complete data view.

[0135] As a preferred embodiment of the present invention, in step (3), feature extraction includes:

[0136] (3-1) Time series feature extraction: Sliding window technology is used to extract the time series features of performance indicator data, including statistical features such as mean μ, standard deviation σ, rate of change r, as well as trend features such as rising / falling trend t, periodic fluctuation p, and mutation point d.

[0137] (3-2) Text feature extraction: The Drain algorithm is used to achieve rapid template extraction from large-scale log data. The Drain algorithm clusters similar log messages through a fixed-depth parse tree to identify log templates and calculates the feature vector V using the TF-IDF (Term Frequency-Inverse Document Frequency) algorithm, where V contains the log keyword weight w, error code distribution e, and operation type frequency f.

[0138] (3-3) Correlation feature extraction: Construct the correlation feature matrix R within the time window T, where the matrix element R[i, j] represents the temporal correlation between the i-th alarm event and the j-th configuration change. The correlation is calculated by the alarm event time t a [i] and configuration change time t c The time difference of [j] is calculated.

[0139] As a preferred embodiment of the present invention, Figure 2 As shown, in step (4), anomaly detection includes:

[0140] (4-1) Statistical anomaly detection: Detect statistical indicators (mean μ, standard deviation σ, rate of change r) in time series features to identify significant deviations of single indicators:

[0141] When |μ-μ base |>3σ base When , the mean is judged to be abnormal;

[0142] When |rr base |>3σ r When , the change rate is judged to be abnormal;

[0143] When|tt base |>3σ t When , the trend is determined to be abnormal;

[0144] where μ base , σ base 、r base , t base are the historical baseline means of the corresponding indicators, obtained through statistical analysis of historical normal data, σ r , σ t is the historical standard deviation of the corresponding indicator, which is used to measure the fluctuation range of the indicator and output the statistical detection result vector ST = [s a , s b , s c ],s a 、s b 、s c Respectively represent the abnormality degree of each statistical indicator;

[0145] (4-2) Deep learning anomaly detection: Construct a joint feature vector F = [F t , F l , F t ], where the time series feature vector is F t =[μ,σ,r,t,p,d], the text feature sub-vector is F l =[w, e, f], the associated eigenvector is F r =[R[i, j]];

[0146] The joint eigenvector F = [F t , F l , F r ] Input autoencoder for anomaly detection:

[0147] The input feature F is compressed into a latent vector Z = Encoder(F) through the encoder, where Encoder is a dimensionality reduction encoding network that compresses high-dimensional features into a low-dimensional latent space;

[0148] Reconstruct the feature F′=Secoder(Z) through the decoder, where Decoder is a reconstruction decoding network that reconstructs the latent vector into the original feature space;

[0149] Calculate the root mean square error err between the input feature F and the reconstructed feature F'. This error reflects the abnormality of the feature. The formula is:

[0150]

[0151] Where i is the feature dimension index. When the reconstruction error err exceeds the preset threshold θ, it is judged as abnormal. The threshold θ is determined based on the reconstruction error distribution of normal samples, usually taking the 95% quantile of the reconstruction error distribution;

[0152] When an anomaly is detected, construct an abnormal feature vector AF = [Z, ST, δ F ], where Z is the potential vector output by the encoder, representing the main mode of the data, ST = [s a , s b , s c ] is the 3-sigma statistical test result, δ F =FF′ is the difference vector between the original feature and the reconstructed feature, which is used to locate abnormal features.

[0153] As a preferred embodiment of the present invention, Figure 3 As shown, in step (5), fault diagnosis includes:

[0154] (5-1) Constructing a knowledge graph:

[0155] The feature library is used to store typical fault features, including the abnormal pattern feature set H of performance indicators, where H includes typical abnormal features of CPU surge pattern, memory leak pattern, and network delay pattern;

[0156] Key log sequence feature set K, where K includes error log patterns, alarm log patterns, and abnormal operation sequence features;

[0157] A set of multi-indicator abnormal association patterns B, where B describes abnormal association rules between multiple performance indicators;

[0158] Case library, used to record actual fault instances, including the time when the fault occurred t s 、Affected range a r , Fault Level s l ;

[0159] The system environment information set Env at the time of the fault, including system configuration, operating status, load conditions, etc.

[0160] The associated mapping set U with the feature library, that is, U[i] indicates which features in the feature library the current fault case has;

[0161] The solution library is used to store fault handling methods, including the standard operating procedure sequence Q for fault handling, the execution condition set X for the handling steps, and the expected result set N;

[0162] A set U of mapping relationships with the case library, where U[i, j] represents the degree to which solution j is applicable to case i, and each mapping has a historical success rate p(y, s), where y represents the failure case, s represents the corresponding solution, and represents the historical success rate of solution s successfully solving failure case y, and an association with the feature library is established through the transfer relationship matrix C=U×U, and its association strength is calculated by w(h, s)×p(y, s), where w(h, s) represents the association weight between feature pattern h and solution s, which is calculated by the number of successful times of the solution corresponding to the feature pattern in historical cases.

[0163] (5-2) Knowledge Graph Reasoning:

[0164] Based on the graph traversal algorithm, the knowledge graph reasoning is performed to detect the abnormal feature vector AF = [Z, ST, δ F ], by calculating each feature pattern h in the feature library i The similarity is:

[0165]

[0166] Among them, h i is the feature vector of the i-th fault mode in the feature library, |AF| and |h i |represents the Euclidean norm of the vector respectively, and the most similar known fault mode is found by calculating the similarity;

[0167] The feature-to-case association strength matrix M is established through the case mapping matrix U. The formula is:

[0168] M[i,j[=U[j][i]×g(h i ), i∈H, j∈Y

[0169] Among them, U[j][i] indicates whether the j-th fault case contains the i-th feature (0 or 1), g(h i ) is the characteristic h i The importance coefficient in fault diagnosis is determined based on the success rate of the feature in historical diagnosis;

[0170] Use graph traversal algorithm to calculate the current anomaly F to historical case y j The accessibility score is:

[0171] score(y j )=∑(sim(AF,h i )×M[i,j]×I(h i )),y j ∈Y

[0172] Where I(h i ) represents the feature h i Impact factor, sim(AF, h i )) represents the similarity between the current abnormal feature and the historical feature pattern, M[i, j] represents the feature h i With case y j The correlation strength, I(h i ) represents the impact factor of feature hi, which is determined based on the position of the feature in the fault propagation link;

[0173] (5-3) Machine Learning Classification:

[0174] The aforementioned joint feature vector AF = [Z, ST, δ F ] is input into the XGBoost classifier for training. The XGBoost classifier constructs a decision tree for each feature dimension, uses second-order Taylor expansion to optimize the loss function, adopts column sampling to reduce overfitting, and outputs the probability distribution P(c|F) of each fault type for subsequent diagnosis result fusion;

[0175] (5-4) Fusion of diagnostic results:

[0176] Perform diagnostic result fusion for each fault type c:

[0177] Calculate the confidence of knowledge graph reasoning, the formula is:

[0178] conf kg (c) = sim(AF, h c )×case_support(c)

[0179] Among them, sim(AF, h c ) is the feature similarity between the current feature and the fault type c, case_support(c) is the historical case support;

[0180] Calculate the confidence of machine learning classification, the formula is:

[0181] conf ml (c)=P(c|F)×V(c)

[0182] Where P(c|F) is the probability output by the XGBoost classifier, and V(c) is the historical accuracy of the fault type;

[0183] S505. Determine the final diagnosis result according to the fusion rules:

[0184] When the knowledge graph reasoning and machine learning classification results are consistent, the consistent fault type is used to retrieve the corresponding solution from the knowledge graph solution library;

[0185] When the results are inconsistent, keep both diagnostic results and construct a diagnostic report, recording the diagnostic source, confidence level, reasoning basis, and recommended solutions for each result;

[0186] As a preferred embodiment of the present invention, Figure 4 As shown, in step (6), fault propagation analysis and root cause location include:

[0187] (6-1) Dependency analysis: Based on the system environment information Env, a dependency directed graph G(N, E) is constructed, where the node set N represents the system components, the edge set E represents the dependency between components, and the edge weight w dep (e i ) represents the dependency strength, where e i Represents the i-th edge in the dependency graph G(N, E);

[0188] For node n that detects an anomaly a , calculate the fault propagation probability, the formula is:

[0189] Prop(n j |n a )=∏w dep (e i ), e i ∈path(n a →n j )

[0190] Among them, path(n a →n j ) indicates that from n a to n j All edges on the path, w dep (e i ) represents the dependency edge e i The weight reflects the dependency strength between components and is determined by the call frequency between components (normalized to 0-1) and the degree of influence of the dependency relationship (0-1 score);

[0191] Construct a fault propagation tree T, which includes all nodes whose fault propagation probability exceeds the threshold δ:

[0192] T=n|Prop(n|na )>δ,n∈N

[0193] Where δ is the preset propagation probability threshold (usually 0.1-0.3), which is used to filter out branches with low propagation probability and retain the main fault propagation path;

[0194] (6-2) Impact Scope Assessment:

[0195] Determine the set of directly affected components D, which includes components directly connected to the fault source node;

[0196] Determine the set of indirectly affected components I, which includes other affected components in the fault propagation tree;

[0197] Calculate the impact for each affected component v:

[0198] impact(v)=s(v)×u(v)×Prop(v|n a )

[0199] Where s(v) is the business importance score of component v (1-5), and u(v) is the percentage of affected users;

[0200] Prioritize based on impact analysis, classify faults according to their impact scope, business importance, and processing complexity, determine processing priorities based on the classification results, and update the analysis results to the case library of the fault knowledge graph.

[0201] As a preferred embodiment of the present invention, in step (7), the process flow optimization includes:

[0202] (7-1) Establishment of process library:

[0203] The processing flow is represented as a directed acyclic graph W(Z, C), where the node set Z represents the operation steps and the edge set C represents the execution order between the steps;

[0204] Construct an execution condition vector for each process, recording the prerequisites required to execute the process, including system environment requirements and resource conditions;

[0205] Build a set of expected results, record the expected troubleshooting results of each process, and only retain results with a high historical success rate;

[0206] (7-2) Effective sequence extraction:

[0207] Extract valid operation sequences from historical records and score each operation step: execution success rate (0-100%), average execution time (minutes), resource consumption level (1-5);

[0208] Based on the comprehensive evaluation of the above three dimensions, the steps are marked as "recommended", "optional" and "avoid";

[0209] (7-3) Critical point identification: steps that meet any of the following conditions are marked as critical decision points: steps with multiple optional branches, checkpoints that require manual judgment, and configuration steps that affect subsequent operations;

[0210] (7-4) Knowledge base update: Evaluate new processing cases: whether they solve the problem, whether they are more efficient than the existing process, and whether they provide new processing ideas. When new cases are proven to be more effective, they are added to the process library.

Claims

1. A deep learning-based adaptive operation and maintenance root cause location method, characterized by: The following steps are involved: S1 collects multi-source data, including system log data, performance indicator data, alarm event data, and configuration change data; S2 performs data cleaning and standardization on the multi-source data, and establishes time series associations between the data; S3 extracts features from the preprocessed data to obtain temporal features, text features, and association features; S4 uses the 3-sigma criterion to perform single-indicator anomaly detection on the statistical indicators in the time series features. At the same time, it constructs the time series features, text features, and association features into a joint feature vector and uses an autoencoder to perform multi-indicator association anomaly detection and output an anomaly feature vector. When an anomaly is detected in S5, reasoning diagnosis is performed based on the fault knowledge graph and combined with the XGBoost classification algorithm for fault diagnosis; S6 performs fault propagation analysis and root cause location on the system based on the fault diagnosis results; S7 updates the fault handling knowledge base based on the root cause location result and generates an optimized standard fault handling process; The feature extraction in S3 includes: S301 uses sliding window technology to extract the time series characteristics of performance indicator data, including statistical characteristics of mean μ, standard deviation σ, rate of change r, as well as trend characteristics of rising / falling trend t, periodic fluctuation p, and mutation point d; S302 uses the Drain algorithm to quickly extract templates from large-scale log data. The Drain algorithm clusters similar log messages through a fixed-depth parse tree to identify log templates and calculates a feature vector V using the TF-IDF (Term Frequency-Inverse Document Frequency) algorithm, where V includes the log keyword weight w, error code distribution e, and operation type frequency f. S303 constructs a correlation feature matrix R within the time window T, where the matrix element R[i, j] represents the temporal correlation between the i-th alarm event and the j-th configuration change. The correlation is calculated by the time difference between the alarm event time ta[i] and the configuration change time tc[j]. The performance indicator data includes: CPU utilization, memory usage, network traffic, disk IO, application response time, number of connections, and queue length.

2. The method for adaptive operation and maintenance root cause location based on deep learning according to claim 1, characterized in that: The step of collecting multi-source data in S1 includes: S101 collects system log data through Agent; S102 collects performance indicator data through SNMP protocol and application monitoring interface; S103 collects alarm event data through SNMP Trap and scheduled polling; S104 collects configuration change data by monitoring configuration file changes and system parameter adjustment records.

3. The method for adaptive operation and maintenance root cause location based on deep learning according to claim 1, characterized in that: The pre-processing step in S2 includes: S201 performs data cleaning on the multi-source data, removes abnormal data and unifies the time format; S202 standardizes the cleaned multi-source data to unify the format and measurement units; S203 establishes a time series association among the performance indicator data, system log data, alarm event data, and configuration change data within the same time window based on the timestamp.

4. The method for adaptive operation and maintenance root cause location based on deep learning according to claim 1, characterized in that: The anomaly detection in S4 includes: S401 performs 3-sigma statistical anomaly detection on the statistical characteristics of a single performance indicator. That is, it detects the statistical indicators in the time series characteristics, namely the mean μ, standard deviation σ, and rate of change r, and identifies significant deviations of a single indicator: When |μ-μ base |>3σ base When , the mean is judged to be abnormal; When |rr base |>3σ r When , the change rate is judged to be abnormal; When|tt base |>3σ t When , the trend is judged to be abnormal; where μ base , σ base 、r base , t base are the historical baseline means of the corresponding indicators, obtained through statistical analysis of historical normal data, σ r , σ t is the historical standard deviation of the corresponding indicator, which is used to measure the fluctuation range of the indicator and output the statistical detection result vector ST = [s a , s b , s c ],s a 、s b 、s c Respectively represent the abnormality degree of each statistical indicator; S402 constructs a joint feature vector F=[F t , F l , F r ], where the time series feature vector is F t =[μ,σ,r,t,p,d], the text feature sub-vector is F l =[w, e, f], the associated eigenvector is F r =[R[i, j]]; S403: Combine the joint feature vector F=[F t , F l , F r ] Input autoencoder for anomaly detection: The input feature F is compressed into a latent vector Z = Encoder(F) through the encoder, where Encoder is a dimensionality reduction encoding network that compresses high-dimensional features into a low-dimensional latent space; Reconstruct the feature F′=Decoder(Z) through the decoder, where Decoder is a reconstruction decoding network that reconstructs the latent vector into the original feature space; Calculate the root mean square error err between the input feature F and the reconstructed feature F'. This error reflects the abnormality of the feature. The formula is: Where i is the feature dimension index. When the reconstruction error err exceeds the preset threshold θ, it is judged as abnormal. The threshold θ is determined based on the reconstruction error distribution of normal samples and the 95% quantile of the reconstruction error distribution is taken. When an anomaly is detected, construct an abnormal feature vector AF = [Z, ST, δ F ], where Z is the potential vector output by the encoder, representing the main mode of the data, ST = [s a , s b , s c ] is the 3-sigma statistical test result, δ F =FF′ is the difference vector between the original feature and the reconstructed feature, which is used to locate abnormal features.

5. The method for adaptive operation and maintenance root cause location based on deep learning according to claim 1, characterized in that: The fault knowledge graph in S5 includes: The feature library is used to store typical fault features, including the abnormal pattern feature set H of performance indicators, where H includes typical abnormal features of CPU surge pattern, memory leak pattern, and network delay pattern; Key log sequence feature set K, where K includes error log patterns, alarm log patterns, and abnormal operation sequence features; A set of multi-indicator abnormal association patterns B, where B describes abnormal association rules between multiple performance indicators; Case library, used to record actual fault instances, including the time when the fault occurred t s 、Affected range a r , Fault Level s l ; The system environment information set Env at the time of the fault, including system configuration, operating status, and load conditions; The associated mapping set U with the feature library, that is, U[i] indicates which features in the feature library the current fault case has; The solution library is used to store fault handling methods, including the standard operating procedure sequence Q for fault handling, the execution condition set X for the handling steps, and the expected result set N; A set U of mapping relationships with the case library, where U[i, j] represents the degree to which solution j is applicable to case i, and each mapping has a historical success rate p(y, s), where y represents the failure case, s represents the corresponding solution, and represents the historical success rate of solution s successfully solving failure case y, and an association with the feature library is established through the transfer relationship matrix C=U×U, and its association strength is calculated by w(h, s)×p(y, s), where w(h, s) represents the association weight between feature pattern h and solution s, which is calculated by the number of successful times of the solution corresponding to the feature pattern in historical cases.

6. The method for adaptive operation and maintenance root cause location based on deep learning according to claim 1, characterized in that: The fault diagnosis in S5 includes: S501 performs knowledge graph reasoning based on graph traversal algorithm: For the abnormal feature vector AF = [Z, ST, δ F ], calculate the similarity with each feature pattern hi in the feature library, the formula is: Among them, h i is the feature vector of the i-th fault mode in the feature library, |F| and |h i |represents the Euclidean norm of the vector respectively, and the most similar known fault mode is found by calculating the similarity; The feature-to-case association strength matrix M is established through the case mapping matrix U. The formula is: {M[i,j]}={U[j][i]×g(h i )},i∈H,j∈Y Among them, U[j][i] indicates whether the j-th fault case contains the i-th feature, and U[j][i] takes the value of 0 or 1. g(h i ) is the importance coefficient of feature hi in fault diagnosis, which is determined based on the success rate of the feature in historical diagnosis; S502 uses a graph traversal algorithm to calculate the reachability score from the current anomaly F to the historical case yj. The formula is: score(y j )=∑(sim(AF,h i )×M[i,j]×I(h i )),y j ∈Y Where I(h i ) represents the feature h i Impact factor, sim(AF, h i ) represents the similarity between the current abnormal feature and the historical feature pattern, M[i, j] represents the feature h i With case y j The correlation strength, I(h i ) represents the feature h i The impact factor of the feature is determined based on the location of the feature in the fault propagation link; S503 uses the XGBoost algorithm for machine learning classification: The aforementioned joint feature vector AF = [Z, ST, δ F ] is input into the XGBoost classifier for training. The XGBoost classifier constructs a decision tree for each feature dimension, uses second-order Taylor expansion to optimize the loss function, adopts column sampling to reduce overfitting, and outputs the probability distribution P(c|F) of each fault type for subsequent diagnosis result fusion; S504 performs diagnostic result fusion for each fault type c: Calculate the confidence of knowledge graph reasoning, the formula is: confkg(c)=sim(AF,h c )×case_support(c) Among them, sim(AF,h c ) is the feature similarity between the current feature and the fault type c, case_suppirt(c) is the historical case support; Calculate the confidence of machine learning classification, the formula is: confml(c)=P(c|F)×V(c) Where P(c|F) is the probability output by the XGBoost classifier, and V(c) is the historical accuracy of the fault type; S505 determines the final diagnosis result according to the fusion rules: When the knowledge graph reasoning and machine learning classification results are consistent, the consistent fault type is used to retrieve the corresponding solution from the knowledge graph solution library; When the results are inconsistent, retain both diagnostic results, construct a diagnostic report, and record the diagnostic source, confidence level, reasoning basis, and recommended solutions for each result.

7. An adaptive operation and maintenance root cause location system based on deep learning, the system is used to implement the adaptive operation and maintenance root cause location method according to any one of claims 1 to 6, characterized in that: Including sequential execution of: The data acquisition and preprocessing module is used to collect and preprocess multi-source data and pass the preprocessed data to the feature extraction module; A feature extraction module is used to receive the pre-processed data to extract features and transmit the extracted features to the anomaly detection module; an anomaly detection module, configured to receive the features and perform anomaly detection, and trigger the fault diagnosis module when an anomaly is detected; A fault diagnosis module is used to diagnose the detected anomaly and output the diagnosis results to the fault propagation analysis module; a fault propagation analysis module, configured to receive the diagnosis results, perform propagation analysis and root cause location, and transmit the analysis results to the process optimization module; A process optimization module is used to establish and optimize a standard fault handling process based on the analysis results. The optimized process is stored and used to guide subsequent fault handling.

Images