Adaptive operation and maintenance root cause positioning method and system based on deep learning

Abstract

The invention discloses a self-adaptive operation and maintenance root cause positioning method and system based on deep learning. The method comprises the steps that firstly, multi-source data such as system logs, performance indexes, alarm events and configuration changes are collected, and time sequence features, text features and associated features are extracted; performing anomaly detection in combination with a 3-sigma criterion and an auto-encoder; when an exception is found, reasoning diagnosis is carried out by utilizing the fault knowledge graph, and fault diagnosis is realized by combining an XGBoost classification algorithm; and then fault propagation analysis and root cause positioning are carried out, and a fault processing flow is continuously optimized. The problems of incomplete fault features, low detection accuracy, limited diagnosis capability and the like in an existing operation and maintenance system are solved, more accurate fault detection and root cause positioning are realized through a mode of combining multi-source data fusion, deep learning detection and knowledge graph reasoning, and meanwhile, the method has adaptive optimization capability and is suitable for large-scale popularization and application. And the operation and maintenance fault processing efficiency is obviously improved.

Description

Technical Field

[0001] The present invention relates to the field of computer system operation and maintenance technology, and in particular to a method and system for adaptive operation and maintenance root cause locating based on deep learning. Background Art

[0002] With the rapid development of new-generation information technologies such as cloud computing, big data, and artificial intelligence, the scale and complexity of enterprise IT infrastructure and application systems are constantly increasing. The dependencies between components in large-scale distributed systems are intricate, and a small failure may cause a chain reaction through the association between systems, leading to a wider range of service anomalies. In this context, how to quickly and accurately locate the root cause of the failure has become a key challenge to ensure the stable operation of the system.

[0003] Traditional operation and maintenance fault handling mainly relies on the experience of operation and maintenance personnel for analysis and diagnosis. Operation and maintenance personnel need to manually collect system logs, monitoring data and other information, and infer possible causes of faults through experience. This method can still cope with simple faults, but when facing modern complex IT systems, there are the following obvious shortcomings: First, manual analysis is inefficient. Faced with massive system data and complex dependencies, manual analysis often takes a lot of time; second, experience judgment is highly subjective, and different operation and maintenance personnel may have large differences in analysis results for the same fault; third, personnel experience is difficult to pass on, and the valuable experience accumulated by excellent operation and maintenance personnel is difficult to effectively precipitate and reuse.

[0004] In recent years, with the development of machine learning technology, the industry has begun to try to apply artificial intelligence technology to the field of fault diagnosis. Some methods identify abnormal patterns by performing text mining on system logs; others use time series analysis techniques to detect abnormal behavior of performance indicators; and others try to use deep learning models to learn the normal behavior patterns of the system. These methods have improved the automation level of fault detection to a certain extent, but there are still many limitations.

[0005] First, in terms of data usage, existing methods are often limited to a single data source. For example, methods that analyze only based on system logs tend to ignore fault information contained in other important data sources such as performance indicators and alarm information; while methods that rely only on performance indicators have difficulty understanding the specific context in which the fault occurred. This limitation of a single data source leads to incomplete fault feature information, affecting the accuracy of diagnosis.

[0006] Secondly, in terms of anomaly detection mechanisms, most methods use relatively simple detection algorithms. Statistical threshold methods are easily affected by data noise; single machine learning models are difficult to cope with complex and changing failure modes. Especially when faced with new types of failures or multiple concurrent failures, these simple detection mechanisms often perform poorly, resulting in a large number of false positives or missed negatives.

[0007] Third, in terms of knowledge management, existing systems generally lack effective management and utilization mechanisms for historical failure experience. Every time a failure occurs, it is necessary to re-analyze it, and it is impossible to fully utilize the historical experience of handling similar failures. This not only wastes resources, but also prolongs the time for troubleshooting.

[0008] Fourth, in terms of root cause location, many methods only stay at the fault detection level, that is, they can find that the system has an abnormality, but it is difficult to further track and locate the specific root cause of the fault. The lack of in-depth analysis of the dependencies between system components makes it difficult to accurately grasp the propagation path of the fault, resulting in the need for large-scale troubleshooting to handle the fault.

[0009] Fifth, in terms of system adaptability, existing methods generally lack the ability to self-optimize and evolve. As new services are launched and system architectures evolve, failure modes are constantly changing. If the diagnostic system cannot adapt to these changes in a timely manner, its diagnostic effectiveness will gradually decrease.

[0010] Therefore, in order to meet the above challenges, a new operation and maintenance root cause location method is urgently needed. This method should be able to integrate multi-source heterogeneous data to obtain comprehensive fault characteristics, use advanced deep learning technology to improve the accuracy of anomaly detection, use knowledge graph technology to accumulate and reuse fault handling experience, achieve accurate root cause location through systematic propagation analysis, and have the ability of continuous learning and adaptive optimization. Summary of the invention

[0011] The purpose of the present invention is to propose an adaptive operation and maintenance root cause location method and system based on deep learning, so as to solve the problems of single data source, low diagnostic accuracy and lack of adaptive optimization in the existing operation and maintenance fault diagnosis, and improve the accuracy and efficiency of operation and maintenance fault root cause location. To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0012] An adaptive operation and maintenance root cause location method based on abnormal feature deep learning and graph reasoning includes the following steps:

[0013] S1 collects multi-source data, where the multi-source data includes system log data, performance indicator data, alarm event data, and configuration change data;

[0014] S2 performs data cleaning and standardization on the multi-source data, and establishes a time series association between the data;

[0015] S3 extracts features from the preprocessed data to obtain time series features, text features, and association features;

[0016] S4 uses the 3-sigma criterion to perform single-indicator anomaly detection on the statistical indicators in the time series features. At the same time, it constructs the time series features, text features and related features into a joint feature vector and uses an autoencoder to perform multi-indicator related anomaly detection and outputs the abnormal feature vector;

[0017] S5 When an anomaly is detected, reasoning diagnosis is performed based on the fault knowledge graph and combined with the XGBoost classification algorithm for fault diagnosis;

[0018] S6 performs fault propagation analysis and root cause location on the system based on the fault diagnosis results;

[0019] S7 updates the fault handling knowledge base based on the root cause location result and generates an optimized standard fault handling process.

[0020] Furthermore, the multi-source data collected in S1 includes: collecting system log data through Agent; collecting performance indicator data through SNMP protocol and application monitoring interface; collecting alarm event data through SNMP Trap and scheduled polling; collecting configuration change data by monitoring configuration file changes and system parameter adjustment records

[0021] Furthermore, the preprocessing step in S2 includes: performing data cleaning on the multi-source data, removing abnormal data and unifying the time format; performing data standardization on the cleaned multi-source data, unifying the format and measurement unit; and establishing a time series association between performance indicator data, system log data, alarm event data and configuration change data within the same time window based on timestamps.

[0022] Furthermore, feature extraction in S3 includes:

[0023] S301. Use sliding window technology to extract the time series characteristics of performance indicator data, including: statistical characteristics of mean μ, standard deviation σ, rate of change r, and trend characteristics of rising / falling trend t, periodic fluctuation p, and mutation point d;

[0024] S302. The Drain algorithm is used to realize fast template extraction of large-scale log data. The Drain algorithm clusters similar log messages through a fixed-depth parse tree to identify log templates, and calculates the feature vector V through the TF-IDF (term frequency-inverse document frequency) algorithm, where V contains the log keyword weight w, error code distribution e, and operation type frequency f;

[0025] S303. Construct a correlation feature matrix R within the time window T, where the matrix element R[i, j] represents the temporal correlation between the i-th alarm event and the j-th configuration change. The correlation is calculated by the alarm event time t a [i] and configuration change time t c The time difference of [j] is calculated.

[0026] Furthermore, anomaly detection in S4 includes:

[0027] S401. Perform 3-sigma statistical anomaly detection on the statistical characteristics of a single performance indicator, that is, detect the statistical indicators (mean μ, standard deviation σ, rate of change r) in the time series characteristics to identify significant deviations of a single indicator:

[0028] When |μ-μ base |>3σ base When , the mean is judged to be abnormal;

[0029] When |rr base |>3σ r When , the change rate is judged to be abnormal;

[0030] When |tt base |>3σ t When , the trend is judged to be abnormal;

[0031] where μ base , base 、r base ,t base are the historical baseline means of the corresponding indicators, obtained through statistical analysis of historical normal data, σ r , σ t is the historical standard deviation of the corresponding indicator, which is used to measure the fluctuation range of the indicator and output the statistical detection result vector ST = [s a ,s b ,s c ],s a 、s b 、s c Respectively represent the abnormal degree of each statistical indicator;

[0032] S402. Construct a joint feature vector F = [F t , F l , F r ], where the time series feature vector is F t =[μ, σ, r, t, p, d], the text feature subvector is F l = [w, e, f], the associated feature vector is F r = [R[i, j]];

[0033] S403: Combine the joint feature vector F=[F t , F l , F r ] Input autoencoder for anomaly detection:

[0034] The input feature F is compressed into a latent vector Z = Encoder(F) through the encoder, where Encoder is a dimensionality reduction encoding network that compresses high-dimensional features into a low-dimensional latent space;

[0035] Reconstruct the feature F′=Decoder(Z) through the decoder, where Decoder is a reconstruction decoding network that reconstructs the latent vector into the original feature space;

[0036] Calculate the root mean square error err between the input feature F and the reconstructed feature F'. This error reflects the abnormality of the feature. The formula is:

[0037]

[0038] Where i is the feature dimension index. When the reconstruction error err exceeds the preset threshold θ, it is judged as abnormal. The threshold θ is determined based on the reconstruction error distribution of normal samples, usually taking the 95% quantile of the reconstruction error distribution;

[0039] When an anomaly is detected, an abnormal feature vector AF = [Z, ST, δ F ], where Z is the latent vector output by the encoder, representing the main mode of the data, ST = [s a ,s b ,s c ] is the 3-sigma statistical test result, δ F =FF′ is the difference vector between the original feature and the reconstructed feature, which is used to locate abnormal features.

[0040] Furthermore, the fault knowledge graph in S5 includes:

[0041] The feature library is used to store typical features of faults, including the abnormal pattern feature set H of performance indicators, where H includes typical abnormal features of CPU surge pattern, memory leakage pattern, and network delay pattern;

[0042] Key log sequence feature set K, where K includes error log pattern, alarm log pattern, and abnormal operation sequence features;

[0043] A set of multi-indicator abnormal association patterns B, where B describes abnormal association rules between multiple performance indicators;

[0044] Case library, used to record actual fault instances, including the time when the fault occurred t s 、Affected Rangea r , Fault Levelsl ;

[0045] The system environment information set Env when the fault occurs, including system configuration, operating status, load conditions, etc.

[0046] The associated mapping set U with the feature library, i.e., U[i] indicates which features in the feature library the current fault case has;

[0047] The solution library is used to store fault handling methods, including the standard operation step sequence Q of fault handling, the execution condition set X of the processing steps, and the expected result set N;

[0048] A set U of mapping relationships with the case library, where U[i, j] represents the degree to which solution j is applicable to case i, and each mapping has a historical success rate p(y, s), where y represents a fault case, s represents the corresponding solution, represents the historical success rate of solution s successfully solving fault case y, and establishes an association with the feature library through a transfer relationship matrix C=U×U, and its association strength is calculated by w(h, s)×p(y, s), where w(h, s) represents the association weight between feature pattern h and solution s, which is calculated by the number of successful times of the solution corresponding to the feature pattern in historical cases.

[0049] Furthermore, the fault diagnosis in S5 includes:

[0050] S501. Knowledge graph reasoning based on graph traversal algorithm:

[0051] For the abnormal feature vector AF = [Z, ST, δ F ], calculate and each feature pattern h in the feature library i The similarity is:

[0052]

[0053] Among them, h i is the feature vector of the ith fault mode in the feature library, |AF| and |h i |represent the Euclidean norm of the vector respectively, and the most similar known fault mode is found by calculating the similarity;

[0054] The feature-to-case association strength matrix M is established through the case mapping matrix U, and the formula is:

[0055] M[i, j] = U[j][i] × g(h i ), i∈H, j∈Y

[0056] Among them, U[j][i] indicates whether the j-th fault case contains the i-th feature (0 or 1), g(h i ) is the characteristic h iThe importance coefficient in fault diagnosis is determined based on the success rate of the feature in historical diagnosis;

[0057] S502. Calculate the current anomaly F to the historical case y using the graph traversal algorithm j The accessibility score is:

[0058] score(y j )=∑(sim(AF,h i )×M[i,j]×I(h i )),y j ∈Y

[0059] Where I(h i ) represents the feature h i The impact factor, sim(AF, h i )) represents the similarity between the current abnormal feature and the historical feature pattern, M[i, j] represents the feature h i With case y j The correlation strength, I(h i ) represents the impact factor of feature hi, which is determined based on the position of the feature in the fault propagation link;

[0060] S503. Use XGBoost algorithm for machine learning classification:

[0061] The above joint feature vector AF = [Z, ST, δ F ] is input into the XGBoost classifier for training. The XGBoost classifier constructs a decision tree for each feature dimension, uses the second-order Taylor expansion to optimize the loss function, uses column sampling to reduce overfitting, and outputs the probability distribution P(c|F) of each fault type for subsequent diagnosis result fusion;

[0062] S504. Perform diagnostic result fusion for each fault type c:

[0063] Calculate the confidence of knowledge graph reasoning, the formula is:

[0064] conf kg (c) = sim(AF, h c )×case_support(c)

[0065] Among them, sim(AF, h c ) is the feature similarity between the current feature and the fault type c, case_support(c) is the historical case support;

[0066] Calculate the confidence of machine learning classification, the formula is:

[0067] conf ml(c)=P(c|F)×V(c)

[0068] Among them, P(c|F) is the probability output by the XGBoost classifier, and V(c) is the historical accuracy of the fault type;

[0069] S505. Determine the final diagnosis result according to the fusion rule:

[0070] When the knowledge graph reasoning and machine learning classification results are consistent, the consistent fault type is used to retrieve the corresponding solution from the solution library of the knowledge graph;

[0071] When the results are inconsistent, keep the two diagnostic results, construct a diagnostic report, and record the diagnostic source, confidence level, reasoning basis, and recommended solutions for each result;

[0072] Furthermore, root cause location in S6 includes:

[0073] Based on the system environment information Env, a dependency directed graph G(N, E) is constructed, where the node set N represents the system components, the edge set E represents the dependency relationship between components, and the edge weight w dep (e i ) represents the dependence strength, where e i Represents the i-th edge in the dependency graph G(N, E);

[0074] For node n that detects an anomaly a , calculate the fault propagation probability, the formula is:

[0075] Prop(n j |n a )=Πw dep (e i ), e i ∈path(n a →n j )

[0076] Among them, path(n a →n j ) indicates that from n a to n j All edges on the path, w dep (e i ) represents the dependency edge e i The weight reflects the dependency strength between components, which is determined by the calling frequency between components (normalized to 0-1) and the influence of the dependency relationship (0-1 score);

[0077] Construct a fault propagation tree T, which contains all nodes whose fault propagation probability exceeds the threshold δ:

[0078] T=n|Prop(n|n a)>δ,n∈N

[0079] Among them, δ is the preset propagation probability threshold (usually 0.1-0.3), which is used to filter out branches with low propagation probability and retain the main fault propagation path;

[0080] S602 evaluates the impact of the fault:

[0081] Determine the set of directly affected components D, which includes components directly connected to the fault source node;

[0082] Determine the indirect impact component set I, which includes other affected components in the fault propagation tree;

[0083] Calculate the impact for each affected component v:

[0084] impact(v)=s(v)×u(v)×Prop(v|n a )

[0085] Where s(v) is the business importance score of component v (1-5), and u(v) is the percentage of affected users;

[0086] S603 performs priority sorting based on impact analysis, classifies faults according to fault impact scope, business importance and processing complexity, determines processing priority based on the classification results, and updates the analysis results to the case library of the fault knowledge graph.

[0087] Furthermore, the optimization of the standard fault handling process in S7 includes:

[0088] S701 Establish a standard process library to store processing procedures:

[0089] The processing flow is represented as a directed acyclic graph W(Z, C), where the node set Z represents the operation steps and the edge set C represents the execution order between the steps;

[0090] Construct an execution condition vector for each process, and record the prerequisites that need to be met to execute the process, including system environment requirements and resource conditions;

[0091] Build a set of expected results, record the expected troubleshooting results of each process, and only keep the results with a high historical success rate;

[0092] S702 extracts valid operation sequence from historical records:

[0093] Score each operation step: execution success rate (0-100%), average execution time (minutes), resource consumption level (1-5);

[0094] Based on the comprehensive evaluation of the above three dimensions, the steps are marked as "recommended", "optional" and "avoid";

[0095] S703 Identify key decision points:

[0096] Mark steps that meet any of the following conditions as key decision points: steps with multiple optional branches, checkpoints that require manual judgment, and configuration steps that affect subsequent operations;

[0097] S704 Update process library:

[0098] Evaluate new processing cases: whether they solve the problem, whether they are more efficient than the existing process, and whether they provide new processing ideas;

[0099] As new cases are proven to be more effective, they are added to the process library.

[0100] The present invention also discloses a system for adaptive root cause location of operation and maintenance based on deep learning of abnormal features and graph reasoning, which is characterized by comprising:

[0101] The data acquisition and preprocessing module is used to acquire and preprocess multi-source data and pass the preprocessed data to the feature extraction module;

[0102] A feature extraction module, used to receive the preprocessed data to extract features and transmit the extracted features to the anomaly detection module;

[0103] an anomaly detection module, used to receive the features and perform anomaly detection, and trigger the fault diagnosis module when an anomaly is detected;

[0104] A fault diagnosis module is used to diagnose the detected anomaly and output the diagnosis result to the fault propagation analysis module;

[0105] A fault propagation analysis module, used to receive the diagnosis result, perform propagation analysis and root cause location, and the analysis result is transmitted to the process optimization module;

[0106] A process optimization module is used to establish and optimize a standard fault handling process according to the analysis results. The optimized process is stored and used to guide subsequent fault handling.

[0107] The beneficial effects of the present invention are as follows:

[0108] (1) This invention innovatively combines 3-sigma statistical detection and deep learning detection based on autoencoders. The 3-sigma criterion is used to identify significant deviations of a single indicator, while the autoencoder discovers abnormal correlations between multiple indicators by feature reconstruction. This dual detection mechanism not only maintains the interpretability of statistical methods, but also uses deep learning to improve the ability to recognize complex abnormal patterns, effectively reducing the false alarm rate.

[0109] (2) The present invention constructs a fault knowledge graph containing a feature library, a case library, and a solution library, and realizes accurate diagnosis by fusing the results of graph reasoning and the XGBoost classification algorithm. The knowledge graph accumulates historical experience, while machine learning provides adaptive capabilities. The combination of the two ensures the accuracy of diagnosis and has the ability to handle new faults.

[0110] (3) The present invention constructs a system component dependency graph to analyze the propagation path and impact range of the fault, thereby achieving accurate root cause location. Compared with the traditional one-by-one troubleshooting method, this graph analysis-based method can quickly identify the source of the fault and assess the impact, greatly improving the efficiency of fault handling.

[0111] (4) The present invention establishes a standard fault handling process library and realizes adaptive evolution of the process through continuous evaluation and optimization. The system continuously improves the processing process by extracting effective operation sequences and identifying key decision points, making fault handling more standardized and efficient, while reducing dependence on personnel experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0112] In order to more clearly illustrate the implementation of the present invention or the existing technical solutions, the following briefly introduces the drawings required to be used in the embodiments or the description of the prior art.

[0113] Figure 1 This is the architecture diagram of the adaptive operation and maintenance root cause positioning system.

[0114] Figure 2 This is the structure diagram of the autoencoder network.

[0115] Figure 3 This is the structure diagram of the fault knowledge graph.

[0116] Figure 4 Propagate dependency graph for failures. DETAILED DESCRIPTION

[0117] The following is a clear and complete description of the technical solutions in the embodiments of the invention in conjunction with the drawings in the embodiments of the invention. Obviously, the described embodiments are only part of the embodiments of the invention, not all of the embodiments. Based on the embodiments of the invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the invention.

[0118] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0119] like Figure 1 As shown, the main steps of the method provided by the present invention are as follows:

[0120] (1) Collecting multi-source data, including system log data, performance indicator data, alarm event data, and configuration change data;

[0121] (2) performing data cleaning and standardization on the multi-source data, and establishing a temporal relationship between the data;

[0122] (3) Extract features from the preprocessed data to obtain temporal features, text features, and association features;

[0123] (4) The 3-sigma criterion is used to perform single-indicator anomaly detection on the statistical indicators in the time series features. At the same time, the time series features, text features and related features are constructed into a joint feature vector and an autoencoder is used to perform multi-indicator related anomaly detection and output the abnormal feature vector;

[0124] (5) When an anomaly is detected, reasoning diagnosis is performed based on the fault knowledge graph and combined with the XGBoost classification algorithm for fault diagnosis;

[0125] (6) Analyze the fault propagation and locate the root cause of the system based on the fault diagnosis results;

[0126] (7) Based on the root cause location result, the fault handling knowledge base is updated and an optimized standard fault handling process is generated.

[0127] As a preferred embodiment of the present invention, in step (1), the process of collecting multi-source data includes: (1-1) collecting system log data through Agent: deploying a lightweight Agent program to collect system operation logs in real time through file reading, system call tracking, etc.;

[0128] (1-2) Collect performance indicator data through SNMP protocol and application monitoring interface: including system-level performance indicators such as CPU utilization, memory usage, network traffic, disk IO, and application-level performance indicators such as application response time, number of connections, and queue length;

[0129] (1-3) Collect alarm event data through SNMP Trap and scheduled polling: Receive SNMPTrap alarms actively reported by devices, and perform scheduled polling to check the alarm status;

[0130] (1-4) Collect configuration change data by monitoring configuration file changes and system parameter adjustment records: including configuration file modifications, system parameter adjustments, software version upgrades and other change information.

[0131] As a preferred embodiment of the present invention, in step (2), data preprocessing includes:

[0132] (2-1) Data cleaning: remove duplicate data, outliers and noise data, and unify data formats and time standards;

[0133] (2-2) Data standardization: standardize the format and unit conversion of data from different sources to achieve data standardization;

[0134] (2-3) Time series association: Establish the association relationship between multi-source data in the same time window to form a complete data view.

[0135] As a preferred embodiment of the present invention, in step (3), feature extraction includes:

[0136] (3-1) Time series feature extraction: The sliding window technology is used to extract the time series features of the performance indicator data, including: statistical features of mean μ, standard deviation σ, rate of change r, as well as trend features of rising / falling trend t, periodic fluctuation p, and mutation point d;

[0137] (3-2) Text feature extraction: The Drain algorithm is used to achieve fast template extraction of large-scale log data. The Drain algorithm clusters similar log messages through a fixed-depth parse tree to identify log templates, and calculates the feature vector V through the TF-IDF (term frequency-inverse document frequency) algorithm, where V contains the log keyword weight w, error code distribution e, and operation type frequency f;

[0138] (3-3) Correlation feature extraction: Construct a correlation feature matrix R within the time window T, where the matrix element R[i, j] represents the temporal correlation between the i-th alarm event and the j-th configuration change. The correlation is calculated by the alarm event time t a [i] and configuration change time t c The time difference of [j] is calculated.

[0139] As a preferred embodiment of the present invention, Figure 2 As shown, in step (4), anomaly detection includes:

[0140] (4-1) Statistical anomaly detection: Detect statistical indicators (mean μ, standard deviation σ, rate of change r) in time series features to identify significant deviations of single indicators:

[0141] When |μ-μ base |>3σ base When , the mean is judged to be abnormal;

[0142] When |rr base |>3σ r When , the change rate is judged to be abnormal;

[0143] When |tt base |>3σ t When , the trend is judged to be abnormal;

[0144] where μ base , σ base 、r base ,t base are the historical baseline means of the corresponding indicators, obtained through statistical analysis of historical normal data, σ r , σ t is the historical standard deviation of the corresponding indicator, which is used to measure the fluctuation range of the indicator and output the statistical detection result vector ST = [s a ,s b ,s c ],s a 、s b 、s c Respectively represent the abnormal degree of each statistical indicator;

[0145] (4-2) Deep learning anomaly detection: construct a joint feature vector F = [F t , F l , F t ], where the time series feature vector is F t =[μ, σ, r, t, p, d], the text feature subvector is F l = [w, e, f], the associated feature vector is F r = [R[i, j]];

[0146] The joint eigenvector F = [F t , F l , F r ] Input autoencoder for anomaly detection:

[0147] The input feature F is compressed into a latent vector Z = Encoder(F) through the encoder, where Encoder is a dimensionality reduction encoding network that compresses high-dimensional features into a low-dimensional latent space;

[0148] Reconstruct the feature F′=Secoder(Z) through the decoder, where Decoder is a reconstruction decoding network that reconstructs the latent vector into the original feature space;

[0149] Calculate the root mean square error err between the input feature F and the reconstructed feature F'. This error reflects the abnormality of the feature. The formula is:

[0150]

[0151] Where i is the feature dimension index. When the reconstruction error err exceeds the preset threshold θ, it is judged as abnormal. The threshold θ is determined based on the reconstruction error distribution of normal samples, usually taking the 95% quantile of the reconstruction error distribution;

[0152] When an anomaly is detected, an abnormal feature vector AF = [Z, ST, δ F ], where Z is the latent vector output by the encoder, representing the main mode of the data, ST = [s a ,s b ,s c ] is the 3-sigma statistical test result, δ F =FF′ is the difference vector between the original feature and the reconstructed feature, which is used to locate abnormal features.

[0153] As a preferred embodiment of the present invention, Figure 3 As shown, in step (5), fault diagnosis includes:

[0154] (5-1) Constructing knowledge graph:

[0155] The feature library is used to store typical features of faults, including the abnormal pattern feature set H of performance indicators, where H includes typical abnormal features of CPU surge pattern, memory leakage pattern, and network delay pattern;

[0156] Key log sequence feature set K, where K includes error log pattern, alarm log pattern, and abnormal operation sequence features;

[0157] A set of multi-indicator abnormal association patterns B, where B describes abnormal association rules between multiple performance indicators;

[0158] Case library, used to record actual fault instances, including the time when the fault occurred t s 、Affected Rangea r , Fault Levels l ;

[0159] The system environment information set Env when the fault occurs, including system configuration, operating status, load conditions, etc.

[0160] The associated mapping set U with the feature library, i.e., U[i] indicates which features in the feature library the current fault case has;

[0161] The solution library is used to store fault handling methods, including the standard operation step sequence Q of fault handling, the execution condition set X of the processing steps, and the expected result set N;

[0162] A set U of mapping relationships with the case library, where U[i, j] represents the degree to which solution j is applicable to case i, and each mapping has a historical success rate p(y, s), where y represents a fault case, s represents the corresponding solution, represents the historical success rate of solution s successfully solving fault case y, and establishes an association with the feature library through a transfer relationship matrix C=U×U, and its association strength is calculated by w(h, s)×p(y, s), where w(h, s) represents the association weight between feature pattern h and solution s, which is calculated by the number of successful times of the solution corresponding to the feature pattern in historical cases.

[0163] (5-2) Knowledge graph reasoning:

[0164] Based on the graph traversal algorithm, the knowledge graph reasoning is performed to detect the abnormal feature vector AF = [Z, ST, δ F ], by calculating each feature pattern h in the feature library i The similarity is:

[0165]

[0166] Among them, h i is the feature vector of the ith fault mode in the feature library, |AF| and |h i |represent the Euclidean norm of the vector respectively, and the most similar known fault mode is found by calculating the similarity;

[0167] The feature-to-case association strength matrix M is established through the case mapping matrix U, and the formula is:

[0168] M[i, j[=U[j][i]×g(h i ), i∈H, j∈Y

[0169] Among them, U[j][i] indicates whether the j-th fault case contains the i-th feature (0 or 1), g(h i ) is the characteristic h i The importance coefficient in fault diagnosis is determined based on the success rate of the feature in historical diagnosis;

[0170] Use graph traversal algorithm to calculate the current anomaly F to historical case y j The accessibility score is:

[0171] score(y j )=∑(sim(AF,h i )×M[i,j]×I(h i )),y j ∈Y

[0172] Where I(h i ) represents the feature h i The impact factor, sim(AF, h i )) represents the similarity between the current abnormal feature and the historical feature pattern, M[i, j] represents the feature h i With case y j The correlation strength, I(h i ) represents the impact factor of feature hi, which is determined based on the position of the feature in the fault propagation link;

[0173] (5-3) Machine Learning Classification:

[0174] The above joint feature vector AF = [Z, ST, δ F ] is input into the XGBoost classifier for training. The XGBoost classifier constructs a decision tree for each feature dimension, uses the second-order Taylor expansion to optimize the loss function, uses column sampling to reduce overfitting, and outputs the probability distribution P(c|F) of each fault type for subsequent diagnosis result fusion;

[0175] (5-4) Fusion of diagnostic results:

[0176] Perform diagnostic result fusion for each fault type c:

[0177] Calculate the confidence of knowledge graph reasoning, the formula is:

[0178] conf kg (c) = sim(AF, h c )×case_support(c)

[0179] Among them, sim(AF, h c ) is the feature similarity between the current feature and the fault type c, case_support(c) is the historical case support;

[0180] Calculate the confidence of machine learning classification, the formula is:

[0181] conf ml (c)=P(c|F)×V(c)

[0182] Among them, P(c|F) is the probability output by the XGBoost classifier, and V(c) is the historical accuracy of the fault type;

[0183] S505. Determine the final diagnosis result according to the fusion rule:

[0184] When the knowledge graph reasoning and machine learning classification results are consistent, the consistent fault type is used to retrieve the corresponding solution from the solution library of the knowledge graph;

[0185] When the results are inconsistent, keep the two diagnostic results, construct a diagnostic report, and record the diagnostic source, confidence level, reasoning basis, and recommended solutions for each result;

[0186] As a preferred embodiment of the present invention, Figure 4 As shown, in step (6), fault propagation analysis and root cause location include:

[0187] (6-1) Dependency analysis: Based on the system environment information Env, a dependency directed graph G(N, E) is constructed, where the node set N represents the system components, the edge set E represents the dependencies between components, and the edge weight w dep (e i ) represents the dependence strength, where e i Represents the i-th edge in the dependency graph G(N, E);

[0188] For node n that detects an anomaly a , calculate the fault propagation probability, the formula is:

[0189] Prop(n j |n a )=∏w dep (e i ), e i ∈path(n a →n j )

[0190] Among them, path(n a →n j ) indicates that from n a to n j All edges on the path, w dep (e i ) represents the dependency edge e i The weight reflects the dependency strength between components, which is determined by the calling frequency between components (normalized to 0-1) and the influence of the dependency relationship (0-1 score);

[0191] Construct a fault propagation tree T, which contains all nodes whose fault propagation probability exceeds the threshold δ:

[0192] T=n|Prop(n|na )>δ,n∈N

[0193] Among them, δ is the preset propagation probability threshold (usually 0.1-0.3), which is used to filter out branches with low propagation probability and retain the main fault propagation path;

[0194] (6-2) Impact Scope Assessment:

[0195] Determine the set of directly affected components D, which includes components directly connected to the fault source node;

[0196] Determine the indirect impact component set I, which includes other affected components in the fault propagation tree;

[0197] Calculate the impact for each affected component v:

[0198] impact(v)=s(v)×u(v)×Prop(v|n a )

[0199] Where s(v) is the business importance score of component v (1-5), and u(v) is the percentage of affected users;

[0200] Prioritize based on impact analysis, classify faults according to fault impact scope, business importance and processing complexity, determine processing priority based on the classification results, and update the analysis results to the case library of the fault knowledge graph.

[0201] As a preferred embodiment of the present invention, in step (7), the process flow optimization includes:

[0202] (7-1) Establishment of process library:

[0203] The processing flow is represented as a directed acyclic graph W(Z, C), where the node set Z represents the operation steps and the edge set C represents the execution order between the steps;

[0204] Construct an execution condition vector for each process, and record the prerequisites that need to be met to execute the process, including system environment requirements and resource conditions;

[0205] Build a set of expected results, record the expected troubleshooting results of each process, and only keep the results with a high historical success rate;

[0206] (7-2) Extraction of effective sequences:

[0207] Extract valid operation sequences from historical records and score each operation step: execution success rate (0-100%), average execution time (minutes), resource consumption level (1-5);

[0208] Based on the comprehensive evaluation of the above three dimensions, the steps are marked as "recommended", "optional" and "avoid";

[0209] (7-3) Critical point identification: Mark steps that meet any of the following conditions as critical decision points: steps with multiple optional branches, checkpoints that require manual judgment, and configuration steps that affect subsequent operations;

[0210] (7-4) Knowledge base update: Evaluate new processing cases to see whether they solve the problem, whether they are more efficient than the existing process, and whether they provide new processing ideas. When new cases are proven to be more effective, they are added to the process library.

Claims

1. An adaptive operation and maintenance root cause location method based on deep learning, characterized in that: The following steps are involved: S1 collects multi-source data, where the multi-source data includes system log data, performance indicator data, alarm event data, and configuration change data; S2 performs data cleaning and standardization on the multi-source data, and establishes a time series association between the data; S3 extracts features from the preprocessed data to obtain time series features, text features, and association features; S4 uses the 3-sigma criterion to perform single-indicator anomaly detection on the statistical indicators in the time series features. At the same time, it constructs the time series features, text features and related features into a joint feature vector and uses an autoencoder to perform multi-indicator related anomaly detection and outputs the abnormal feature vector; S5 When an anomaly is detected, reasoning diagnosis is performed based on the fault knowledge graph and combined with the XGBoost classification algorithm for fault diagnosis; S6 performs fault propagation analysis and root cause location on the system based on the fault diagnosis results; S7 updates the fault handling knowledge base based on the root cause location result and generates an optimized standard fault handling process.

2. According to claim 1, the method for adaptive operation and maintenance root cause location based on deep learning is characterized in that: The step of collecting multi-source data in S1 includes: S101. Collect system log data through Agent; S102. Collect performance indicator data through SNMP protocol and application monitoring interface; S103. Collect alarm event data through SNMP Trap and timed polling; S104. Collect configuration change data by monitoring configuration file changes and system parameter adjustment records.

3. The method for adaptive operation and maintenance root cause location based on deep learning according to claim 1, characterized in that: The preprocessing step in S2 includes: S201. Clean the multi-source data, remove abnormal data and unify the time format; S202. Standardize the cleaned multi-source data and unify the format and measurement unit; S203. Establish a time series association among the performance indicator data, system log data, alarm event data and configuration change data within the same time window based on the timestamp.

4. The method for adaptive operation and maintenance root cause location based on deep learning according to claim 1, characterized in that: The feature extraction in S3 includes: S301. Use sliding window technology to extract the time series characteristics of performance indicator data, including: statistical characteristics of mean μ, standard deviation σ, rate of change r, and trend characteristics of rising / falling trend t, periodic fluctuation p, and mutation point d; S302. The Drain algorithm is used to realize fast template extraction of large-scale log data. The Drain algorithm clusters similar log messages through a fixed-depth parse tree to identify log templates, and calculates the feature vector V through the TF-IDF (term frequency-inverse document frequency) algorithm, where V contains the log keyword weight w, error code distribution e, and operation type frequency f; S303. Construct a correlation feature matrix R within the time window T, where the matrix element R[i, j] represents the temporal correlation between the i-th alarm event and the j-th configuration change. The correlation is calculated by the alarm event time t a [i] and configuration change time t c The time difference of [j] is calculated.

5. The method for adaptive operation and maintenance root cause location based on deep learning according to claim 1, characterized in that: The abnormality detection in S4 includes: S401. Perform 3-sigma statistical anomaly detection on the statistical characteristics of a single performance indicator, that is, detect the statistical indicators (mean μ, standard deviation σ, rate of change r) in the time series characteristics to identify significant deviations of a single indicator: When |μ-μ base |>3σ base When , the mean is judged to be abnormal; When |rr base |>3σ r When , the change rate is judged to be abnormal; When |tt base |>3σ t When , the trend is judged to be abnormal; where μ base , σ base 、r base ,t base are the historical baseline means of the corresponding indicators, obtained through statistical analysis of historical normal data, σ r , σ t is the historical standard deviation of the corresponding indicator, which is used to measure the fluctuation range of the indicator and output the statistical detection result vector ST = [s a ,s b ,s c ],s a 、s b 、s c Respectively represent the abnormal degree of each statistical indicator; S402. Construct a joint feature vector F=[F t , F l , F r ], where the time series feature vector is F t =[μ, σ, r, t, p, d], the text feature subvector is F l = [w, e, f], the associated feature vector is F r = [R[i, j]]; S403: Combine the joint feature vector F=[F t , F l , F r ] Input autoencoder for anomaly detection: The input feature F is compressed into a latent vector Z = Encoder(F) through the encoder, where Encoder is a dimensionality reduction encoding network that compresses high-dimensional features into a low-dimensional latent space; Reconstruct the feature F′=Decoder(Z) through the decoder, where Decoder is a reconstruction decoding network that reconstructs the latent vector into the original feature space; Calculate the root mean square error err between the input feature F and the reconstructed feature F'. This error reflects the abnormality of the feature. The formula is: Where i is the feature dimension index. When the reconstruction error err exceeds the preset threshold θ, it is judged as abnormal. The threshold θ is determined based on the reconstruction error distribution of normal samples, usually taking the 95% quantile of the reconstruction error distribution; When an anomaly is detected, an abnormal feature vector AF = [Z, ST, δ F ], where Z is the latent vector output by the encoder, representing the main mode of the data, ST = [s a ,s b ,s c ] is the 3-sigma statistical test result, δ F =FF′ is the difference vector between the original feature and the reconstructed feature, which is used to locate abnormal features.

6. The method for adaptive operation and maintenance root cause location based on deep learning according to claim 1, characterized in that: The fault knowledge graph in S5 includes: The feature library is used to store typical features of faults, including the abnormal pattern feature set H of performance indicators, where H includes typical abnormal features of CPU surge pattern, memory leakage pattern, and network delay pattern; Key log sequence feature set K, where K includes error log pattern, alarm log pattern, and abnormal operation sequence features; A set of multi-indicator abnormal association patterns B, where B describes abnormal association rules between multiple performance indicators; Case library, used to record actual fault instances, including the time when the fault occurred t s 、Affected Rangea r , Fault Levels l ; The system environment information set Env when the fault occurs, including system configuration, operating status, load conditions, etc. The associated mapping set U with the feature library, i.e., U[i] indicates which features in the feature library the current fault case has; The solution library is used to store fault handling methods, including the standard operation step sequence Q of fault handling, the execution condition set X of the processing steps, and the expected result set N; A set U of mapping relationships with the case library, where U[i, j] represents the degree to which solution j is applicable to case i, and each mapping has a historical success rate p(y, s), where y represents a fault case, s represents the corresponding solution, represents the historical success rate of solution s successfully solving fault case y, and establishes an association with the feature library through a transfer relationship matrix C=U×U, and its association strength is calculated by w(h, s)×p(y, s), where w(h, s) represents the association weight between feature pattern h and solution s, which is calculated by the number of successful times of the solution corresponding to the feature pattern in historical cases.

7. The method for adaptive operation and maintenance root cause location based on deep learning according to claim 1, characterized in that: The fault diagnosis in S5 includes: S501. Knowledge graph reasoning based on graph traversal algorithm: For the abnormal feature vector AF = [Z, ST, δ F ], calculate and each feature pattern h in the feature library i The similarity is: Among them, h i is the feature vector of the ith fault mode in the feature library, |F| and |h i |represent the Euclidean norm of the vector respectively, and the most similar known fault mode is found by calculating the similarity; The feature-to-case association strength matrix M is established through the case mapping matrix U, and the formula is: M[i,j]=U[j][i]×g(h i ),i∈H,j∈Y Among them, U[j][i] indicates whether the j-th fault case contains the i-th feature (0 or 1), g(h i ) is the characteristic h i The importance coefficient in fault diagnosis is determined based on the success rate of the feature in historical diagnosis; S502. Calculate the current anomaly F to the historical case y using the graph traversal algorithm j The accessibility score is: score(y j )=∑(sim(AF,h i )×M[i,j]×I(h i )),y j ∈Y Where I(h i ) represents the feature h i The impact factor, sim(AF, h i )) represents the similarity between the current abnormal feature and the historical feature pattern, M[i, j] represents the feature h i With case y j The correlation strength, I(h i ) represents the impact factor of feature hi, which is determined based on the position of the feature in the fault propagation link; S503. Use XGBoost algorithm for machine learning classification: The above joint feature vector AF = [Z, ST, δ F ] is input into the XGBoost classifier for training. The XGBoost classifier constructs a decision tree for each feature dimension, uses the second-order Taylor expansion to optimize the loss function, uses column sampling to reduce overfitting, and outputs the probability distribution P(c|F) of each fault type for subsequent diagnosis result fusion; S504. Perform diagnostic result fusion for each fault type c: Calculate the confidence of knowledge graph reasoning, the formula is: conf kg (c)=sim(AF,h c )×case_support(c) Among them, sim(AF, h c ) is the feature similarity between the current feature and the fault type c, case_suppirt(c) is the historical case support; Calculate the confidence of machine learning classification, the formula is: conf ml (c)=P(c|F)×V(c) Among them, P(c|F) is the probability output by the XGBoost classifier, and V(c) is the historical accuracy of the fault type; S505. Determine the final diagnosis result according to the fusion rule: When the knowledge graph reasoning and machine learning classification results are consistent, the consistent fault type is used to retrieve the corresponding solution from the solution library of the knowledge graph; When the results are inconsistent, keep the two diagnostic results, construct a diagnostic report, and record the diagnostic source, confidence level, reasoning basis, and recommended solutions for each result.

8. The method for adaptive operation and maintenance root cause location based on deep learning according to claim 1, characterized in that: The root cause location in S6 includes: S602. Construct a dependency directed graph G(N, E) based on the system environment information Env, where the node set N represents the system components, the edge set E represents the dependency between components, and the edge weight w dep (e i ) represents the dependence strength, where e i Represents the i-th edge in the dependency graph G(N, E); For node n that detects an anomaly a , calculate the fault propagation probability, the formula is: Prop(n j |n a )=∏w dep (its i ),its i ∈good(n a →n j ) Among them, path(n a →n j ) indicates that from n a to n j All edges on the path, w dep (e i ) represents the dependency edge e i The weight reflects the dependency strength between components, which is determined by the calling frequency between components (normalized to 0-1) and the influence of the dependency relationship (0-1 score); Construct a fault propagation tree T, which contains all nodes whose fault propagation probability exceeds the threshold δ: T=n|Prop(n|n a )>δ,n∈N Among them, δ is the preset propagation probability threshold (usually 0.1-0.3), which is used to filter out branches with low propagation probability and retain the main fault propagation path; S602. Assess the scope of the fault impact: Determine the set of directly affected components D, which includes components directly connected to the fault source node; Determine the indirect impact component set I, which includes other affected components in the fault propagation tree; Calculate the impact for each affected component v: impact(v)=s(v)×u(v)×Prop(v|n a ) Where s(v) is the business importance score of component v (1-5), and u(v) is the percentage of affected users; S603. Prioritize based on impact analysis, classify faults according to fault impact scope, business importance and processing complexity, determine processing priority based on the classification results, and update the analysis results to the case library of the fault knowledge graph.

9. The method for adaptive operation and maintenance root cause location based on deep learning according to claim 1, characterized in that: The optimization of the standard fault handling process in S7 includes: S701 Establish a standard process library to store processing procedures: The processing flow is represented as a directed acyclic graph W(Z, C), where the node set Z represents the operation steps and the edge set C represents the execution order between the steps; Construct an execution condition vector for each process, and record the prerequisites that need to be met to execute the process, including system environment requirements and resource conditions; Build a set of expected results, record the expected troubleshooting results of each process, and only keep the results with a high historical success rate; S702 extracts valid operation sequence from historical records: Score each operation step: execution success rate (0-100%), average execution time (minutes), resource consumption level (1-5); Based on the comprehensive evaluation of the above three dimensions, the steps are marked as "recommended", "optional" and "avoid"; S703 Identify key decision points: Mark steps that meet any of the following conditions as key decision points: steps with multiple optional branches, checkpoints that require manual judgment, and configuration steps that affect subsequent operations; S704 Update process library: Evaluate new processing cases: whether they solve the problem, whether they are more efficient than the existing process, and whether they provide new processing ideas; As new cases are proven to be more effective, they are added to the process library.

10. An adaptive operation and maintenance root cause location system based on deep learning, characterized in that: Including sequential execution: The data acquisition and preprocessing module is used to acquire and preprocess multi-source data and pass the preprocessed data to the feature extraction module; A feature extraction module, used to receive the preprocessed data to extract features and transmit the extracted features to the anomaly detection module; an anomaly detection module, used to receive the features and perform anomaly detection, and trigger the fault diagnosis module when an anomaly is detected; A fault diagnosis module is used to diagnose the detected anomaly and output the diagnosis result to the fault propagation analysis module; A fault propagation analysis module, used to receive the diagnosis result, perform propagation analysis and root cause location, and the analysis result is transmitted to the process optimization module; A process optimization module is used to establish and optimize a standard fault handling process according to the analysis results. The optimized process is stored and used to guide subsequent fault handling.

Images