Abnormal data location method based on integrated anomaly detector and root cause analysis

By integrating an abnormality detector and root cause analysis, the problem of lack of real-time root cause analysis in equipment abnormality detection is solved, and the rapid positioning and prevention and maintenance of abnormal data is achieved, which improves operation and maintenance efficiency and equipment reliability.

CN114168374BActive Publication Date: 2024-11-29THE 28TH RES INST OF CHINA ELECTRONICS TECH GROUP CORP

Patent Information

Application Number
CN202111440027.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-30
Publication Date
2024-11-29
Estimated Expiration
2041-11-30

AI Technical Summary

Technical Problem

The existing technology lacks real-time root cause analysis capabilities in equipment abnormality detection, which makes it difficult for operation and maintenance personnel to prevent and maintain in a timely manner, increasing the risk of equipment failure.

Method used

Using a method based on the integrated exception detector and root cause analysis, data preprocessing and feature extraction are performed through integrated supervision, semi-supervised and unsupervised exception detectors, and anomaly detection and root cause analysis are achieved using majority voting method and association rule mining algorithm. , provided to operation and maintenance personnel to ensure the safe and reliable operation of the equipment.

Benefits of technology

It improves the robustness and accuracy of abnormal detection, helps operation and maintenance personnel to quickly identify abnormal information, ensure early failure recovery, transform passive maintenance to active maintenance, and ensure the stable operation of equipment resources.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114168374B_ABST
    Figure CN114168374B_ABST
Patent Text Reader

Abstract

The present invention proposes an abnormal data detection method based on the combination of an integrated anomaly detector and root cause analysis. After the integrated anomaly detector is used to detect the abnormality of the equipment, the root cause analysis algorithm is immediately triggered, and the root cause analysis algorithm is used to identify the specific dimensions that cause the abnormality, thereby realizing abnormal root cause analysis. The abnormal data location method proposed in the present invention integrates supervision, semi-supervision and unsupervised learning algorithms to realize comprehensive detection of known abnormal types and unknown abnormal types, which can largely avoid the occurrence of abnormal false alarms or omissions due to inaccurate results of a single anomaly detection model. Anomaly detection can automatically discover problems, and root cause location can further locate the specific dimensions of the problem based on the discovery of the problem, which is convenient for operation and maintenance personnel to quickly locate the fault direction and improve the efficiency of troubleshooting.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of anomaly analysis, and in particular relates to an abnormal data locating method based on the combination of an integrated anomaly detector and root cause analysis. Background Art

[0002] Anomaly detection technology is widely used in practical fields such as credit card fraud, network intrusion detection, complex industrial equipment, medicine, status monitoring of space shuttle systems, image and video, etc. By using anomaly detection technology, credit card fraud, malicious activity intrusion in computers, abnormal operation status of industrial equipment, etc. can be discovered, and then appropriate measures can be taken to protect the safety of users or equipment and improve its reliability. Today, in the era of big data, the computing power of computers has been greatly improved, and data-driven anomaly detection technology has become more and more popular and applied to various tasks. It has excellent performance and has great practical application value.

[0003] At present, the main anomaly detection technologies include supervised anomaly detection technology, semi-supervised anomaly detection technology, and unsupervised anomaly detection technology. The performance of each machine learning algorithm is determined by its sample data, and has an important relationship with the sample dimension, sample size, sample quality, sample label, etc. Different anomaly detection algorithms have different performance in different sample spaces. At present, most anomaly detection work is implemented by using a single anomaly detection technology. The output result of a single anomaly detector directly determines the quality of anomaly detection work, which is one-sided. Summary of the invention

[0004] Purpose of the invention: The technical problem to be solved by the present invention is how to analyze the operating status of the equipment based on the monitoring data of various types of equipment resources, obtain the health status of the equipment in real time, and immediately locate the root cause of the abnormality when an equipment abnormality is detected, so as to help operation and maintenance personnel to perform preventive maintenance on the equipment and avoid irreversible consequences caused by the failure of a certain equipment resource; and how to combine intelligent algorithms with traditional operation and maintenance to assist in anomaly detection and root cause analysis, so as to quickly improve the operation and maintenance efficiency, and provide an abnormal data locating method based on the combination of integrated anomaly detector and root cause analysis.

[0005] In order to solve the above technical problems, the present invention discloses an abnormal data location method based on the combination of an integrated anomaly detector and root cause analysis, comprising:

[0006] Step 1, perform anomaly detection on a network equipment monitoring data set to obtain a detection result; the anomaly detection is performed using an integrated anomaly detector, and the integrated anomaly detector is obtained by integrated learning of a supervised anomaly detector, a semi-supervised anomaly detector and an unsupervised anomaly detector; the detection result includes a normal result and an abnormal result; the detection models established under multiple anomaly detection criteria are integrated into one, and multiple detection results are comprehensively considered. By using an integrated strategy, the robustness of the detection model can be improved and the detection quality can be improved.

[0007] Step 2: Root cause analysis is performed on the abnormal results to obtain the root cause of the abnormality and locate the abnormal data. The integrated anomaly detector is combined with the root cause analysis algorithm to trigger the root cause analysis after the anomaly is detected. The root cause analysis algorithm is used to quickly locate the attribute value combination that causes the abnormality in the multi-dimensional attribute space and provide it to the operation and maintenance personnel for preventive maintenance to ensure the safe and reliable operation of the equipment.

[0008] In one implementation, step 1 includes:

[0009] Step 1-1, according to the requirements of the integrated anomaly detector for the network device monitoring data set, complete the allocation of the network device monitoring data set; the network device monitoring data set includes a training set and a test set;

[0010] Step 1-2: Preprocess the network equipment monitoring data set;

[0011] Step 1-3: Perform feature extraction on the preprocessed data to obtain extracted data;

[0012] Step 1-4: Input the extracted data into the supervised anomaly detector, semi-supervised anomaly detector and unsupervised anomaly detector in the integrated anomaly detector for training and testing respectively;

[0013] Step 1-5: For the test set, make an integrated decision on the output results of the supervised anomaly detector, semi-supervised anomaly detector and unsupervised anomaly detector to obtain the detection results; complete the anomaly identification of the test set and realize integrated anomaly detection.

[0014] In one implementation, the supervised anomaly detector in step 1 uses the extreme learning machine algorithm (ELM), the semi-supervised anomaly detector uses the one-class extreme learning machine algorithm (OC-ELM), and the unsupervised anomaly detector uses the isolation forest algorithm (Iforest). All possibilities are considered comprehensively through ensemble learning, effectively reducing the sensitivity of model performance to a single anomaly detection criterion.

[0015] In one implementation, the network device monitoring data set allocation in step 1-1 includes dividing the data set, training set allocation, and test set allocation;

[0016] The data attributes of the network device monitoring data set include network response time, CPU, memory, bandwidth utilization, throughput, packet forwarding rate and port traffic;

[0017] In order to verify the effectiveness of the proposed integrated anomaly detector, the network equipment monitoring data set is divided into five categories of data, recorded as category 0 to category 4, where category 0 represents normal data, which is collected when the network equipment is in normal operation; category 1 represents abnormal type 1 data, category 2 represents abnormal type 2 data, and abnormal type 1 data and abnormal type 2 data are collected when the network equipment is in two situations where abnormal operation often occurs; category 3 represents abnormal type 3 data, category 4 represents abnormal type 4 data, and abnormal type 3 data and abnormal type 4 data are collected when the network equipment is in two situations where abnormal operation occasionally occurs; in addition, the two types of data of abnormal type 1 and abnormal type 2 are used as data of known abnormal types, and the two types of data of abnormal type 3 and abnormal type 4 are used as data of unknown abnormal types;

[0018] The training set is allocated as follows:

[0019] For the extreme learning machine algorithm, the training set consists of normal class data, abnormal type 1, and abnormal type 2 data;

[0020] For the single-class extreme learning machine algorithm, the training set consists only of the same normal class data as in the training set of the extreme learning machine algorithm;

[0021] For the Isolation Forest algorithm, no training process is required;

[0022] The test set is allocated as follows:

[0023] Test the test sets of unknown anomalies and known anomalies respectively to verify the effectiveness of the model;

[0024] Known anomaly detection: The test sets of the three anomaly detectors are the same, consisting of normal type, anomaly type 1, and anomaly type 2 data.

[0025] Unknown anomaly detection: The test sets of the three anomaly detectors are the same, consisting of normal type, anomaly type 3, and anomaly type 4 data.

[0026] In one implementation, the data preprocessing in step 1-2 includes data standardization processing, and the data standardization processing adopts the Z-score standardization method to obtain standardized new data by calculating the mean and variance of the original data. The new data obeys a normal distribution with a mean of 0 and a standard deviation of 1; the new data x * Calculated by the following formula:

[0027] x * =(x-μ) / σ

[0028] Among them, μ is the mean of the original data x, and σ is the standard deviation of the original data.

[0029] In one implementation, the data preprocessing in step 1-2 also includes data denoising, and the data denoising uses a moving average filtering method to smooth and denoise the data.

[0030] In one implementation, in steps 1-3, a principal component analysis algorithm PCA (Principal Component Analysis) is used to extract features from multidimensional data, and representative and effective components are selected to form a new feature vector.

[0031] In one implementation, the majority voting method is used in steps 1-5 to make an integrated decision on the output results of the supervised anomaly detector, the semi-supervised anomaly detector and the unsupervised anomaly detector to obtain the detection results of the test data in the test set, and the detection results include normal results or abnormal results.

[0032] In one implementation, in step 2, an association rule mining algorithm is used to implement root cause analysis. The association rule mining algorithm can identify frequently occurring data attribute sets from a test set of a network device monitoring data set. The frequently occurring data attribute sets are the root causes of anomalies, thereby locating abnormal data.

[0033] In one implementation, the association rule mining algorithm in step 2 is an Apriori algorithm, and for a test set of network equipment monitoring data sets, a frequently occurring data attribute set is found through continuous iteration, and the frequently occurring data attribute set is the root cause of the anomaly.

[0034] Beneficial effects:

[0035] The present invention adopts an integrated learning algorithm to realize anomaly detection. By combining the results of multiple anomaly detection algorithms through integrated learning, the diversity of the integrated anomaly detection model can make up for the shortcomings of each anomaly detection algorithm in specific fields. After the anomaly is triggered, the root cause analysis is used to locate the abnormal attributes, helping the operation and maintenance personnel to identify the abnormal information as early as possible, ensuring the rapid recovery of early faults, transforming passive into active maintenance, and ensuring the stable and normal operation of equipment resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more clear.

[0037] Figure 1 Schematic diagram of anomaly detection algorithm classification.

[0038] Figure 2 A flowchart of the method provided in the embodiment of the present application.

[0039] Figure 3 A flowchart of step 1 of the method provided in an embodiment of the present application.

[0040] Figure 4 Schematic diagram of data set allocation in step 1-1 of the method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0042] The embodiment of the present application proposes an abnormal data locating method based on the combination of an integrated anomaly detector and root cause analysis, which can be applied to the anomaly locating scenario of network equipment monitoring data. Data preprocessing is performed based on the monitoring data of the network equipment to obtain a network equipment monitoring data set, and anomaly detection is performed on the network equipment monitoring data set to discover abnormal information of network equipment resources; the root cause analysis is used to locate the abnormal attributes, helping operation and maintenance personnel to identify abnormal information as early as possible, ensuring rapid recovery of early faults, transforming passive into active maintenance, and ensuring stable and normal operation of network equipment resources.

[0043] The present application embodiment discloses a method for locating abnormal data based on the combination of an integrated anomaly detector and root cause analysis, such as Figure 2 As shown, including:

[0044] Step 1, performing anomaly detection on a network device monitoring data set to obtain a detection result; the anomaly detection is performed using an integrated anomaly detector, and the integrated anomaly detector is obtained by integrated learning of a supervised anomaly detector, a semi-supervised anomaly detector and an unsupervised anomaly detector; the detection result includes a normal result and an abnormal result;

[0045] Step 2: Analyze the root cause of the abnormal results, obtain the root cause of the abnormality, and locate the abnormal data.

[0046] Anomaly detection algorithms are mainly implemented through unsupervised, semi-supervised and supervised methods, such as Figure 1 shown.

[0047] The supervised anomaly detection algorithm needs to divide the data set into a training set and a test set. Both the training set and the test set need to contain normal and abnormal data and be labeled. First, the training set is used to train and learn the detection model, and then the test set is used to test the performance. The supervised anomaly detection model is similar to the commonly used classification and recognition model, but the categories in anomaly detection are unbalanced. The supervised anomaly detection algorithm alleviates the impact of imbalance by improving the existing classification algorithm. However, the supervised technology needs to assume that the anomaly is known. Only by training on the known anomaly class can the anomaly be correctly diagnosed. This technology requires the ability to correctly label normal and abnormal labels. However, in many actual application scenarios, the abnormality of the device is not known in advance and may occur spontaneously during the testing phase.

[0048] The dataset of the semi-supervised anomaly detection algorithm consists of a training set and a test set, but the training set only contains normal data of the device operation. The algorithm mainly builds a detection model by learning the characteristics of normal class samples, describing the normal operation status of the device; in the test phase, by inputting the test sample into the detection model, comparing the deviation degree between the test sample and the learned model to give the corresponding anomaly score, and then comparing the anomaly score with the set threshold to determine whether the test sample is abnormal. Semi-supervised technology implements anomaly detection based on a single classification framework.

[0049] The unsupervised anomaly detection algorithm is the most flexible of the three methods. It does not require data labels or division into training and test sets. It uses the intrinsic information of the data set to judge instances that deviate from the majority of the data as anomalies. Typical anomaly detection algorithms include density-based and distance-based algorithm estimation. Anomaly detection in practical applications is usually aimed at large unlabeled data. It is more convenient and more widely used to use unsupervised algorithms for anomaly detection.

[0050] In summary, during the training process, the supervised anomaly detection algorithm will learn a sample set of known anomaly types, and match the test samples according to the learned knowledge base, and can only complete the detection of known anomaly types. In actual applications, the anomalies that occur in the device are usually unknown, and it is extremely difficult and unrealistic to obtain all the abnormal modes that may occur in the device. Therefore, it is necessary to use an unsupervised or semi-supervised anomaly detector to identify unknown abnormal patterns. However, when detecting known anomalies, the accuracy of unsupervised and semi-supervised detectors is lower than that of supervised anomaly detectors. In summary, the embodiment of the present application proposes an integrated anomaly detector that integrates a supervised anomaly detection algorithm, a semi-supervised anomaly detection algorithm, and an unsupervised anomaly detection algorithm, which realizes the comprehensive detection of known anomalies and unknown anomalies. The integrated anomaly detector proposed in the embodiment of the present application comprehensively makes up for the shortcomings of the three types of algorithms in specific fields, and has diversity and robustness.

[0051] In this embodiment, in step 1, the supervised anomaly detector uses the extreme learning machine algorithm, the semi-supervised anomaly detector uses the single-classification extreme learning machine algorithm, and the unsupervised anomaly detector uses the isolation forest algorithm.

[0052] In this embodiment, the flowchart of step 1 is as follows Figure 3 As shown, including:

[0053] Step 1-1: according to the requirements of different anomaly detection algorithms in the integrated anomaly detector for data sets, complete the allocation of network device monitoring data sets; the network device monitoring data sets include training sets and test sets;

[0054] The data attributes of the network device monitoring data set include network response time, CPU, memory, bandwidth utilization, throughput, packet forwarding rate and port traffic;

[0055] ① In order to verify the effectiveness of the proposed integrated anomaly detector, it is necessary to divide the network equipment monitoring data set into 5 categories of data, among which category 0 represents normal data, which is collected when the network equipment is in normal operation; category 1 represents abnormal type 1 data, category 2 represents abnormal type 2 data, and abnormal type 1 data and abnormal type 2 data are collected when the network equipment is in two situations where abnormal operation often occurs. In this embodiment, abnormal type 1 data is collected when the network equipment has an abnormal operation state with a routing configuration error, and abnormal type 2 data is collected when the network equipment has an abnormal operation state with an excessively high routing load; category 3 represents abnormal type 3 data, category 4 represents abnormal type 4 data, and abnormal type 3 data and abnormal type 4 data are collected when the network equipment is in two situations where abnormal operation occasionally occurs. In this embodiment, abnormal type 3 data is collected when the network equipment has an abnormal operation state with a line fault, and abnormal type 4 data is collected when the network equipment has an SNMP (Simple Network Management The data is collected under the abnormal operation state when the Simple Network Management Protocol (SMP) process is closed; in addition, the two types of data of abnormal type 1 and abnormal type 2 are used as the data of known abnormal type, and the two types of data of abnormal type 3 and abnormal type 4 are used as the data of unknown abnormal type. The division of the network equipment monitoring data set is shown in the figure below. Figure 4 shown.

[0056] ② Training process, the training set is allocated as follows:

[0057] For the extreme learning machine ELM algorithm, the training set consists of normal class data, abnormal type 1, and abnormal type 2 data;

[0058] For the single-class extreme learning machine OC-ELM algorithm, the training set consists only of the same normal class data as in the training set of the extreme learning machine algorithm;

[0059] For the Isolation Forest algorithm, no training process is required;

[0060] ③Test process, the test set is allocated as follows:

[0061] The test sets with unknown anomalies and known anomalies are tested separately to verify the effectiveness of the model.

[0062] Known anomaly detection: The test sets of the three anomaly detectors are the same, consisting of normal samples, anomaly type 1, and anomaly type 2 data.

[0063] Unknown anomaly detection: The test sets of the three anomaly detectors are the same, consisting of normal samples, anomaly type 3, and anomaly type 4 data.

[0064] The two types of data, anomaly type 1 and anomaly type 2, are trained and learned by the supervised learning algorithm, which requires a large amount of data and a high diagnosis rate. Therefore, these two types of data are collected and obtained in two abnormal situations where network equipment often occurs; anomaly type 3 and anomaly type 4 represent the occurrence of unknown anomalies and do not need to participate in the training process. In actual situations, the amount of data is generally small. Therefore, they are collected and obtained in two situations where network equipment is in an occasionally abnormal operating state.

[0065] Step 1-2: Data preprocessing, including data standardization, data noise reduction, etc.

[0066] ① Data standardization uses the Z-score standardization method to obtain standardized new data by calculating the mean and variance of the original data. The new data obeys a normal distribution with a mean of 0 and a standard deviation of 1. * Calculated by the following formula:

[0067] x * =(x-μ) / σ

[0068] Among them, μ is the mean of the original data x, and σ is the standard deviation of the original data.

[0069] ②Data denoising: Use moving average filtering method to smooth and denoise the data.

[0070] Step 1-3: Feature extraction: use the principal component analysis (PCA) algorithm to extract features from multidimensional data and select representative and effective components to form a new feature vector;

[0071] Step 1-4: Input the extracted data into the supervised anomaly detector, semi-supervised anomaly detector and unsupervised anomaly detector in the integrated anomaly detector for training and testing respectively;

[0072] Step 1-5: For the test set, the majority voting method is used to make an integrated decision on the output results of the three types of anomaly detectors to obtain the detection results of the test data in the test set, and the detection results include normal results or abnormal results.

[0073] In this embodiment, step 2 is to combine the integrated anomaly detector with the root cause analysis algorithm. After the anomaly is detected by the integrated anomaly detector, the root cause analysis algorithm is used to analyze the data set to find out the specific attributes that cause the anomaly, so as to help the operation and maintenance personnel identify the root cause of the anomaly and perform preventive maintenance as soon as possible to avoid more serious equipment failures. In this embodiment, an association rule mining algorithm is used to implement root cause analysis. The association rule mining algorithm can identify frequently occurring data attribute sets from the test set of the network equipment monitoring data set. The frequently occurring data attribute sets are the root causes of the anomaly, thereby locating the abnormal data.

[0074] The association rule algorithm in this embodiment adopts the Apriori algorithm, and through continuous iteration, finds the largest frequent attribute set as the root cause of abnormal data to achieve root cause analysis. The Apriori algorithm is to find the final frequent item set by continuously scanning the test set of the entire network device monitoring data set and calculating the support of all item sets. For example, in the network device monitoring data set, select the abnormal type 2 data (i.e., the data set with excessively high routing load) as the test set. In step 1, the detection result is an abnormal result, then in step 2, select the abnormal data set with excessively high routing load. The frequent attribute set related to the excessively high routing load situation mined by the Apriori algorithm is excessively high CPU, excessively high packet loss rate, etc. Therefore, the abnormal cause of the positioning may be that the amount of data passing through the router is too large at this time, and the attribute indicators such as CPU and packet loss rate caused by excessively large carrying traffic are abnormal, which ultimately leads to excessively high routing load.

[0075] The present invention provides an abnormal data location method based on the combination of an integrated anomaly detector and root cause analysis. There are many methods and ways to implement the technical solution. The above is only a specific implementation of the present invention. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present invention, several improvements and modifications can be made, and these improvements and modifications should also be regarded as the protection scope of the present invention. All components not specified in this embodiment can be implemented by existing technologies.

Claims

1. A method for locating abnormal data based on the combination of integrated anomaly detector and root cause analysis, characterized in that: include: Step 1: Perform anomaly detection on the network device monitoring data set to obtain the detection result; The anomaly detection is performed by using an integrated anomaly detector, which is obtained by integrated learning of a supervised anomaly detector, a semi-supervised anomaly detector and an unsupervised anomaly detector; the detection result includes a normal result and an abnormal result; Step 2: Analyze the root cause of the abnormal result, obtain the root cause of the abnormality, and locate the abnormal data; Step 1 includes: Step 1-1, according to the requirements of the integrated anomaly detector for the network device monitoring data set, complete the allocation of the network device monitoring data set; the network device monitoring data set includes a training set and a test set; Step 1-2: Preprocess the network equipment monitoring data set; Step 1-3: Perform feature extraction on the preprocessed data to obtain extracted data; Steps 1-4: Input the extracted data into the supervised anomaly detector and semi-supervised anomaly detector in the integrated anomaly detector for training and testing, and input it into the unsupervised anomaly detector for testing; Step 1-5: For the test set, make an integrated decision on the output results of the supervised anomaly detector, the semi-supervised anomaly detector and the unsupervised anomaly detector to obtain the detection results; In step 1, the supervised anomaly detector adopts the extreme learning machine algorithm, the semi-supervised anomaly detector adopts the single-classification extreme learning machine algorithm, and the unsupervised anomaly detector adopts the isolation forest algorithm; The network device monitoring data set allocation in step 1-1 includes dividing the data set, training set allocation and test set allocation; The data attributes of the network device monitoring data set include network response time, CPU, memory, bandwidth utilization, throughput, packet forwarding rate and port traffic; The network equipment monitoring data set is divided into 5 categories of data, recorded as category 0 to category 4, where category 0 represents normal data, which is collected when the network equipment is in normal operation; category 1 represents abnormal type 1 data, category 2 represents abnormal type 2 data, and abnormal type 1 data and abnormal type 2 data are collected when the network equipment is in two situations where abnormal operation often occurs; category 3 represents abnormal type 3 data, category 4 represents abnormal type 4 data, and abnormal type 3 data and abnormal type 4 data are collected when the network equipment is in two situations where abnormal operation occasionally occurs; in addition, the two types of data of abnormal type 1 and abnormal type 2 are regarded as data of known abnormal types, and the two types of data of abnormal type 3 and abnormal type 4 are regarded as data of unknown abnormal types; The training set is allocated as follows: For the extreme learning machine algorithm, the training set consists of normal class data, abnormal type 1, and abnormal type 2 data; For the single-class extreme learning machine algorithm, the training set consists only of the same normal class data as in the training set of the extreme learning machine algorithm; For the Isolation Forest algorithm, no training process is required; The test set is distributed as follows: Test the test sets of unknown anomalies and known anomalies separately. Known anomaly detection: The test sets of the three anomaly detectors are the same, consisting of normal type, anomaly type 1, and anomaly type 2 data; Unknown anomaly detection: The test sets of the three anomaly detectors are the same, consisting of normal type, anomaly type 3, and anomaly type 4 data; In the steps 1-5, the majority voting method is used to make an integrated decision on the output results of the supervised anomaly detector, the semi-supervised anomaly detector and the unsupervised anomaly detector to obtain the detection results of the test data in the test set, and the detection results include normal results or abnormal results; In step 2, an association rule mining algorithm is used to implement root cause analysis. The association rule mining algorithm can identify frequently occurring data attribute sets from a test set of a network device monitoring data set. The frequently occurring data attribute sets are the root causes of anomalies, thereby locating abnormal data.

2. The method for locating abnormal data based on the combination of integrated anomaly detector and root cause analysis according to claim 1, characterized in that: The data preprocessing in step 1-2 includes data standardization processing, and the data standardization processing adopts the Z-score standardization method to obtain standardized new data by calculating the mean and variance of the original data. The new data obeys a normal distribution with a mean of 0 and a standard deviation of 1; the new data Calculated by the following formula: , in, For the original data x The mean of is the standard deviation of the original data.

3. The method for locating abnormal data based on the combination of integrated anomaly detector and root cause analysis according to claim 2, characterized in that: The data preprocessing in step 1-2 also includes data denoising, and the data denoising uses a moving average filtering method to smooth and denoise the data.

4. The method for locating abnormal data based on the combination of integrated anomaly detector and root cause analysis according to claim 1, characterized in that: In the steps 1-3, the principal component analysis algorithm PCA is used to extract features from the multidimensional data to obtain feature vectors.

5. The method for locating abnormal data based on the combination of integrated anomaly detector and root cause analysis according to claim 1, characterized in that: The association rule mining algorithm in step 2 is the Apriori algorithm. For the test set of the network equipment monitoring data set, a frequently occurring data attribute set is found through continuous iteration. The frequently occurring data attribute set is the root cause of the anomaly.

Citation Information

Patent Citations

  • An abnormality detection method of injection molding machine blocking based on ensemble learning

    CN109145948A

  • Single-index anomaly detection method based on fusion of multiple unsupervised methods

    CN111507376A

Cited By

  • Mobile base station complaint root cause analysis method based on Bayesian network

    CN116226336A

  • Root cause analysis method for mobile base station complaints based on Bayesian network

    CN116226336B