A method for discovering root causes of failures based on storage device model

By using the combination of connectivity graphs, alarm root cause analysis graphs and root cause derivation graphs in the storage device system for norm calculations, and combining the FP-Growth algorithm to construct the root cause derivation graphs, the problem of failure root cause positioning uncertainty in the storage device system is solved, and higher positioning accuracy and better performance are achieved.

CN114385399BActive Publication Date: 2025-03-14BEIJING ZZNODE TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202210017636.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-01-07
Publication Date
2025-03-14
Estimated Expiration
2042-01-07

AI Technical Summary

Technical Problem

In a CMDB-based storage device system, the root of the failure has uncertain factors due to the location of the fault, and there may be missed or false alarms of hardware abnormal indicators, which will affect the derivation results and lead to the location error of the root cause.

Method used

The root cause discovery method based on the storage device model is used, and the connected graph, alarm root cause analysis graph and root cause derivation graph are used for norm calculations. The root cause derivation graph is constructed through the FP-Growth correlation analysis algorithm to reduce the dependence on manual prior knowledge, and mainly rely on historical data for rule mining.

Benefits of technology

Improve the accuracy and performance of fault cause positioning, reduce labor costs for operation and maintenance personnel, and show better performance in complex storage device models.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114385399B_ABST
    Figure CN114385399B_ABST
Patent Text Reader

Abstract

A method for discovering root causes of faults based on a storage device model, which performs norm calculation by utilizing a combination of a connectivity graph, an alarm root cause analysis graph, and a root cause deduction graph, can be helpful in constructing a root cause deduction graph by reading historical records, using a data abstraction method and an FP‑Growth algorithm, and the obtained root cause deduction graph is output as a root cause discovery rule. When a server runs in real time and an alarm occurs, the root cause deduction graph used in the present invention is used as a basis for judgment. Compared with a fault uncertainty graph corresponding to a fault case, the root cause deduction graph used in the present invention occupies less space and has better performance in a complex storage device model, while reducing dependence on artificial prior knowledge, mainly relying on historical data for rule mining, and saving the labor cost of operation and maintenance personnel.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a fault alarm technology of a computer storage device system, in particular to a fault root cause discovery method based on a storage device model. Background Art

[0002] Root cause location of faults is an important part of ensuring the reliability and security of storage device systems. When a node in a storage device system fails, the root cause of the fault must be quickly located to effectively restore the storage device system. Configuration Management Database (CMDB) is a database that digitally stores enterprise IT assets. CMDB can store a large amount of asset information such as hardware, software, and applications within the enterprise, as well as the complex relationships between these asset information. These asset information, as the underlying cornerstone, supports other operation and maintenance systems to jointly ensure the continuity and health of the enterprise's business. How to quickly find the root cause of the CMDB-based data storage structure alarm is of self-evident importance.

[0003] In the related art, when a fault occurs, the abnormal indicators that cause the fault alarm are obtained; and a derivation graph is constructed based on the obtained abnormal indicators, and the root cause corresponding to the alarm is determined based on the matching degree between the derivation graph and the fault uncertainty graph corresponding to the fault case in the established fault case library. Among them, the abnormal indicators include an abnormal indicator of a device within the first and second specified time periods when the alarm occurs, and the fault derivation graph refers to two nodes on any edge with a direction representing two abnormalities among the abnormal indicators obtained, and the probability on any edge is used to characterize the probability that an abnormal indicator causes another abnormal indicator. The uncertainty graph of each fault case in the fault case library corresponds to the determined root cause of the fault. However, in actual applications, due to the particularity of the hardware topology structure based on the CMDB storage device system, there are certain uncertainties in the search for the root cause of the fault, and some or all of the abnormal indicators of some hardware may be missed or misreported. This situation will greatly affect the derivation results and lead to errors in root cause location. Summary of the invention

[0004] The present invention aims at the defects or shortcomings in the prior art and provides a method for discovering the root cause of a fault based on a storage device model.

[0005] The technical solution of the present invention is as follows:

[0006] A method for discovering root causes of faults based on a storage device model is characterized in that a connectivity graph, an alarm root cause analysis graph, and a root cause derivation graph are used to calculate norms according to the following formula:

[0007]

[0008] Where d is the norm, s is the number of frequent items in the connected graph, and p y i It represents the probability value of the previous fault leading to the next fault in the alarm root cause analysis diagram, p i Represents the root cause derivation diagram with p y i The probability value that the previous abnormal indicator between nodes with the same connected fault node causes another abnormal indicator;

[0009] Select k alarm root cause analysis graphs with the smallest norm value, where k is a positive integer greater than 1, determine the graph structure with the same node name as the alarm root cause analysis graph from the root cause deduction graph, calculate the probability of k different graph structures appearing in historical data, select the graph structure with the highest probability as the final result of the alarm root cause analysis graph, and obtain the final root cause of this alarm.

[0010] The k=2.

[0011] The implementation of the root cause derivation diagram includes the following steps:

[0012] Step A: Collect historical data, including system logs, performance data, and system alarm information;

[0013] Step B, preprocessing the collected historical data;

[0014] Step C, abstracting the preprocessed historical data;

[0015] Step D, using the abstracted historical data to obtain the nodes of the root cause derivation graph according to the FP-Growth association analysis algorithm, and constructing the FP-Tree graph structure of the root cause derivation graph;

[0016] Step E, calculate the probability of the previous item causing the next item between each pair of frequent set items, and obtain the connection direction between the two nodes in the root cause deduction graph according to the probability value, until a complete root cause deduction graph is obtained.

[0017] The performance data in step A includes disk information, disk domain information, storage pool information, and host group information obtained based on the CMDB storage device model; the system alarm information includes CPU alarms, memory alarms, PCIE I / O device alarms and power alarms obtained based on the CMDB storage device model; the system log includes client ID, client IP, protocol ID and file system ID obtained based on the CMDB storage device model.

[0018] The disk information includes disk ID, disk occupancy and disk health status; the disk domain information includes disk domain ID, disk domain occupancy, hot spare disk capacity and disk domain health status; the storage pool information includes storage pool ID, storage pool occupancy and main memory type; the host group information contains host group ID and host capacity utilization; the CPU alarm includes CPU configuration alarm, CPU voltage regulator alarm, CPU core temperature over-limit alarm and CPU initialization alarm; the memory alarm includes memory configuration error alarm and memory temperature over-limit alarm; the PCIE I / O device alarm includes PCIe error alarm reported by BIOS system and I / O channel check error alarm; the power supply alarm includes power supply device voltage over-limit alarm.

[0019] The step B includes missing value processing, outlier processing, and discrete data encoding.

[0020] The step C includes extracting all the above data within five minutes of the alarm occurring in the historical data, and abstracting the extracted data into a combination of fault ID and fault topology location id according to the topological structure position of the alarm, and its attribute is the fault alarm time, wherein the fault topology location id is composed of the host group id, storage pool id, disk domain id and disk id. If the fault occurs in the disk domain, the fault topology location id is composed of the group id, storage pool id and disk domain id, and the remaining spaces are filled with 0s. The fault topology location id of the host group fault alarm is composed of the host group id, and the remaining positions are filled with 0s.

[0021] The step D includes using the FP-Growth association analysis algorithm to obtain frequent set items that appear within a 5-minute time period as nodes of the root cause analysis diagram. When alarms occur at multiple topological structure locations of the server, a corresponding number of FP-Tree diagrams are generated according to the number of locations.

[0022] The step E includes calculating the connection relationship of the root cause derivation graph composed of frequent set items. In the root cause derivation graph, two nodes on any edge with a direction represent the combination of the device name and the fault ID of each abnormal indicator obtained. Any edge with a direction is used to indicate the association relationship between the abnormal indicators represented by the two nodes on the edge. The probability on any edge is used to represent the probability p(a, b) that an abnormal indicator a causes another abnormal indicator b. The specific probability calculation method is as follows:

[0023]

[0024] Where s(a, b) indicates the number of times an alarm of an abnormal indicator a causes an alarm of another abnormal indicator b, a indicates the number of times alarm a occurs, and n indicates the total number of alarm data.

[0025] According to the actual situation of the server and the difference in alarm sensitivity, there may be two situations: the phenomenon alarm is earlier than the root cause alarm, and the root cause alarm is earlier than the phenomenon alarm. Here, the connection direction of the root cause deduction diagram is determined according to the specific server properties. The alarm type at the starting point is the effect relative to the alarm type at the end point, that is, the phenomenon alarm type, and the alarm type at the end point is the cause relative to the alarm type at the starting point, that is, the root cause alarm.

[0026] Use the following steps to create a connectivity diagram and alarm root cause analysis diagram:

[0027] Step 1: Obtain alarm data: When an alarm occurs on the server, obtain the server's real-time data within five minutes, including system logs, performance data, system alarm information, and server hardware topology, and perform data abstraction on the obtained alarm data;

[0028] Step 2: Generate an alarm root cause analysis tree: Use the FP-Growth algorithm to obtain frequent items from all the alarm data. The frequent items obtained are the collection of root cause fault alarms and more frequent phenomenon alarms caused by the root cause. Calculate the probability that the first alarm causes the second alarm between the frequent items, and save all the connected graphs obtained.

[0029] The technical effects of the present invention are as follows: A method for discovering root causes of faults based on a storage device model of the present invention performs norm calculation by utilizing a combination of a connectivity graph, an alarm root cause analysis graph, and a root cause deduction graph. This method can be beneficial for constructing a root cause deduction graph by reading historical records, using data abstraction and an FP-Growth algorithm. The obtained root cause deduction graph is output as a root cause discovery rule. When the server runs in real time and an alarm occurs, the root cause deduction graph used in the present invention is used as a basis for judgment. Compared with the fault uncertainty graph corresponding to the fault case, the root cause deduction graph used in the present invention occupies less space and has better performance in complex storage device models. At the same time, it reduces dependence on artificial prior knowledge, mainly relies on historical data for rule mining, and saves the labor cost of operation and maintenance personnel. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 The present invention is a schematic diagram of an alarm root cause analysis process for implementing a method for discovering root causes of faults based on a storage device model. Figure 1 The method includes step 1, collecting historical data; step 2, data abstraction; step 3, rule mining; step 4, generating a fault root cause analysis diagram based on obtaining real-time fault alarms; step 5, correcting parameters and connection relationships; and step 6, obtaining the alarm root cause.

[0031] Figure 2 It is a schematic diagram of the storage device structure exemplified in a method for discovering root causes of faults based on a storage device model according to the present invention. Figure 2From top to bottom, the diagram includes application servers, host groups, LUN groups (LUN, Logical Unit Number), storage pools, disk domains, and hard disks. Application servers include Windows application servers and Linux application servers. Host groups include host group 1 and host group 2. LUN groups include LUN group 1 and LUN group 2. Storage pools include the first group of data areas / hot spare areas, the second group of data areas / hot spare areas, and the third group of data areas / hot spare areas, which are distributed from left to right. Disk domains include disk domain 1, disk domain 2, and disk domain 3, which are distributed from left to right. Hard disks include SSD (SSD, Solid State Disk), SAS (SAS, Serial Attach SCSI), and NL-SAS (NL-SAS, Near Line Serial Attach SCSI). SSD and disk domain 1 form a high-performance area, SAS and disk domain 2 form a performance area, and NL-SAS and disk domain 3 form a capacity area.

[0032] Figure 3 The present invention is a schematic diagram of the hardware structure of a method for discovering the root cause of a fault based on a storage device model. Figure 3 The computer device includes a network interface, a memory, and a storage interconnected with a processor respectively, and the memory includes a fault root cause locating device. DETAILED DESCRIPTION

[0033] Below is the attached figure ( Figure 1-Figure 3 ) and Examples illustrate the present invention.

[0034] Figure 1 The present invention is a schematic diagram of an alarm root cause analysis process for implementing a method for discovering root causes of faults based on a storage device model. Figure 2 It is a schematic diagram of the storage device structure exemplified in a method for discovering root causes of faults based on a storage device model according to the present invention. Figure 3 This is a schematic diagram of the hardware structure of a method for discovering the root cause of a fault based on a storage device model according to the present invention. Figures 1 to 3 As shown, a fault root cause discovery method based on a storage device model is characterized in that the norm calculation is performed according to the following formula using a connectivity graph, an alarm root cause analysis graph, and a root cause derivation graph:

[0035]

[0036] Where d is the norm, s is the number of frequent items in the connected graph, and p y i It represents the probability value of the previous fault leading to the next fault in the alarm root cause analysis diagram, p i Represents the root cause derivation diagram with p y iThe probability value of the previous abnormal indicator causing another abnormal indicator between the nodes with the same connected fault nodes is represented; select k alarm root cause analysis graphs with the smallest norm value, k is a positive integer greater than 1, determine the graph structure with the same node name as the alarm root cause analysis graph from the root cause derivation graph, calculate the probability of k different graph structures appearing in historical data, select the graph structure with the highest probability as the final result of the alarm root cause analysis graph, and obtain the final root cause of this alarm. The k=2.

[0037] The implementation of the root cause derivation graph includes the following steps: step A, collecting historical data, including system logs, performance data and system alarm information; step B, preprocessing the collected historical data; step C, abstracting the preprocessed historical data; step D, using the abstracted historical data to obtain the nodes of the root cause derivation graph according to the FP-Growth association analysis algorithm, and constructing the FP-Tree graph structure of the root cause derivation graph; step E, calculating the probability of the previous item causing the next item between the frequent set items, and obtaining the connection direction between the two nodes in the root cause derivation graph according to the probability value, until a complete root cause derivation graph is obtained.

[0038] The performance data in step A includes disk information, disk domain information, storage pool information, and host group information obtained based on the CMDB storage device model; the system alarm information includes CPU alarms, memory alarms, PCIE I / O device alarms and power alarms obtained based on the CMDB storage device model; the system log includes client ID, client IP, protocol ID and file system ID obtained based on the CMDB storage device model. The disk information includes disk ID, disk occupancy and disk health status; the disk domain information includes disk domain ID, disk domain occupancy, hot spare disk capacity and disk domain health status; the storage pool information includes storage pool ID, storage pool occupancy and main memory type; the host group information contains host group ID and host capacity utilization; the CPU alarm includes CPU configuration alarm, CPU voltage regulator alarm, CPU core temperature over-limit alarm and CPU initialization alarm; the memory alarm includes memory configuration error alarm and memory temperature over-limit alarm; the PCIE I / O device alarm includes PCIe error alarm reported by BIOS system and I / O channel check error alarm; the power supply alarm includes power supply device voltage over-limit alarm.

[0039] The step B includes missing value processing, outlier processing, and discrete data encoding. The step C includes extracting all the above data within five minutes of the alarm in the historical data, and abstracting the extracted data into a combination of fault ID and fault topology location id according to the topological structure position of the alarm, and its attribute is the fault alarm time, wherein the fault topology location id is composed of the host group id, storage pool id, disk domain id and disk id. If the fault occurs in the disk domain, the fault topology location id is composed of the group id, storage pool id and disk domain id, and the remaining vacancies are filled with 0. The fault topology location id of the host group fault alarm is composed of the host group id, and the remaining positions are filled with 0. The step D includes using the FP-Growth association analysis algorithm to obtain frequent set items that appear within a 5-minute time period as nodes of the root cause analysis diagram. When alarms occur in multiple topological structure locations of the server, a corresponding number of FP-Tree diagrams are generated according to the number of locations.

[0040] The step E includes calculating the connection relationship of the root cause derivation graph composed of frequent set items. In the root cause derivation graph, two nodes on any edge with a direction represent the combination of the device name and the fault ID of each abnormal indicator obtained. Any edge with a direction is used to indicate the association relationship between the abnormal indicators represented by the two nodes on the edge. The probability on any edge is used to represent the probability p(a, b) that an abnormal indicator a causes another abnormal indicator b. The specific probability calculation method is as follows:

[0041]

[0042] Among them, s(a, b) indicates the number of times an alarm of an abnormal indicator a causes an alarm of another abnormal indicator b, a indicates the number of times alarm a occurs, and n indicates the total number of alarm data. According to the actual situation of the server and the difference in alarm sensitivity, there will be two situations: the phenomenon alarm is earlier than the root cause alarm and the root cause alarm is earlier than the phenomenon alarm. Here, the connection direction of the root cause derivation graph is determined according to the specific server attributes. The alarm type at the starting point is the result relative to the alarm type at the end point, that is, the phenomenon alarm type, and the alarm type at the end point is the cause relative to the alarm type at the starting point, that is, the root cause alarm.

[0043] The connectivity graph and alarm root cause analysis graph are realized through the following steps: Step 1, obtain alarm data: When an alarm occurs on the server, obtain the real-time data of the server within five minutes, including system logs, performance data and system alarm information, as well as the server hardware topology, and perform data abstraction on the obtained alarm data; Step 2, generate the alarm root cause analysis tree: Use the FP-Growth algorithm to obtain frequent set items from all the alarm data obtained. The obtained frequent set items are the collection of root cause fault alarms and more frequent phenomenon alarms caused by the root cause, and calculate the probability that the first alarm causes the second alarm between the obtained frequent set items, and save all the obtained connectivity graphs.

[0044] The present invention relates to a method for discovering root causes of storage device model failures based on CMDB, wherein the storage device structure is as follows: Figure 2 The overall root cause analysis process is as follows: Figure 1 As shown, the LUN (Logical Unit Number, LUN) group represents a logical disk group created on a storage area network storage. The present invention belongs to a method for obtaining a graph structure by mapping real-time fault alarm data and finding the root cause of the fault by correcting the connection relationship and coefficient, and includes the following two parts.

[0045] The first part is historical rule discovery, the method includes:

[0046] Step 1: Data extraction: Collect historical data, including system logs, performance data, and system alarm information. Based on the CMDB storage device model, the performance data obtained are disk information, including disk id, disk occupancy and disk health status; disk domain information, including disk domain id, disk domain occupancy, hot spare disk capacity and disk domain health status; storage pool information, including storage pool id, storage pool occupancy and main memory type (including SSD (Solid State Disk, SSD), SAS (Serial Attach SCSI, SAS), and NL-SAS (Near Line-Serial Attach SCSI, NL-SAS); host group information, including host group id and host capacity utilization. System alarms are hardware alarms of host groups in the storage device model based on CMDB. They are divided into: CPU alarms, memory alarms, PCIE I / O device alarms and power alarms. Among them, CPU alarms include CPU configuration alarms, CPU voltage regulator alarms, CPU core temperature over-limit alarms, CPU initialization alarms, etc. Memory alarms include memory configuration error alarms and memory temperature over-limit alarms. PCIE device and I / O device alarms include BIOS system (BasicInputOutput The system log of the storage device model entity based on CMDB is to store shared client information, including client id, client ip, protocol id and file system id.

[0047] In particular, the data sources required to be collected are xx server alarm records and system logs, and this application does not restrict the data source.

[0048] Step 2: Data preprocessing: The historical data collected above are cleaned and preprocessed, and the preprocessing includes necessary preprocessing processes such as missing value processing, outlier processing, and discrete data encoding.

[0049] Step 3: Abstract the acquired data. Extract all the above data within five minutes of the alarm in the historical data. According to the topological structure location of the alarm, abstract the extracted data into a combination of fault ID and fault topological location ID, and its attribute is the fault alarm time. Among them, the fault topological location ID is composed of host group ID, storage pool ID, disk domain ID and disk ID.

[0050] It should be noted that if the fault occurs in the disk domain, the fault topology location id consists of the group id, storage pool id and disk domain id, and the remaining spaces are filled with 0. The fault topology location id of the host group fault alarm consists of the host group id, and the remaining spaces are filled with 0.

[0051] Step 4: Use the FP-Growth (Frequent Pattern Growth, FP-Growth) association analysis algorithm to obtain the frequent set items that appear within a 5-minute period as the nodes of the root cause analysis diagram. Construct the FP-Tree (Frequent Pattern Tree, FP-Tree) graph structure. Optionally, when alarms occur in multiple topological locations of the server, a corresponding number of FP-Tree graphs will be generated based on the number of locations.

[0052] Step 5: Calculate the connection relationship of the root cause derivation graph composed of the frequent set items obtained in step 4. In the root cause derivation graph, the two nodes on any edge with a direction represent the combination of the device name and fault ID of each abnormal indicator obtained, and any edge with a direction is used to indicate the association relationship between the abnormal indicators represented by the two nodes on the edge. The probability on any edge is used to characterize the probability p that one abnormal indicator causes another abnormal indicator; the specific probability calculation method is as follows.

[0053]

[0054] Indicates the probability that alarm b will occur after alarm a occurs, where:

[0055]

[0056] In the formula, a represents the number of times alarm a appears in the alarm data obtained in step 3, and n represents the total number of alarm data obtained in step 3. Thus, the probability of the previous item causing the next item between two frequent set items is calculated, and the connection direction between two nodes in the root cause deduction graph is obtained according to the probability value. Finally, a complete root cause deduction graph is obtained.

[0057] By reading historical records, using data abstraction and the FP-Growth algorithm, a root cause deduction graph is constructed. The resulting root cause deduction graph is output as a root cause discovery rule, which serves as a basis for judgment when the server runs in real time and an alarm occurs.

[0058] In particular, according to the actual situation of the server and the difference in alarm sensitivity, there will be two situations: the phenomenon alarm is earlier than the root cause alarm and the root cause alarm is earlier than the phenomenon alarm. Here, the connection direction of the root cause derivation graph is performed according to the specific server attributes. The alarm type at the starting end is the effect relative to the alarm type at the end end, that is, the phenomenon alarm type, and the alarm type at the end end is the cause relative to the alarm type at the starting end, that is, the root cause alarm.

[0059] Part 2: Obtain real-time data for root cause analysis.

[0060] Step 1: Get alarm data. When an alarm occurs on the server, get the real-time data of the server within five minutes, including system logs, performance data, system alarm information, and server hardware topology. And use the method of step 3 in historical rule mining to abstract the acquired alarm data.

[0061] Step 2: Generate alarm root cause analysis tree: Use FP-Growth algorithm to obtain frequent set items from all alarm data. The obtained frequent set items are the collection of root cause fault alarms and more frequent phenomenon alarms caused by the root cause. According to the probability calculation method of step 5 in historical rule mining, the probability of the first alarm causing the second alarm between the obtained frequent set items is calculated, and all the obtained connected graphs are saved.

[0062] In particular, when the alarm obtained is contrary to the server alarm rules, the saved graph will be deleted. For example, in the historical data, it is found that in the CPU fault alarm, the phenomenon alarm time always occurs before the root cause alarm time. In the collected real-time data, the graph structure whose node time pattern is contrary to it will be deleted, and the remaining graph will be retained as the alarm root cause analysis graph.

[0063] Optionally, if alarms of multiple hardware structures occur in the server, root cause analysis is performed on the alarms of each structure separately.

[0064] Step 3: Correct the connection relationship. The connected graph saved in step 2 is used to calculate the norm of the alarm root cause analysis graph obtained by historical rule mining and the root cause derivation graph with the same frequent set items. The specific calculation method is as follows:

[0065]

[0066] Where s represents the number of frequent items, p y i Indicates the probability value of the previous fault leading to the next fault in the alarm root cause analysis diagram, p i Represents the root cause derivation diagram with p y iThe probability value of the previous abnormal indicator causing another abnormal indicator between nodes with the same connected fault nodes. The smaller the norm value d is, the closer the alarm root cause analysis diagram is to the root cause derivation diagram, and the closer the structure of the alarm root cause analysis diagram is to the real result. We select the k alarm root cause analysis diagrams with the smallest norm value as the output of this step. k is a variable parameter, which is set to 2 here.

[0067] Step 4: Correct the connection relationship. Since the root cause derivation graph contains a graph structure with the same node name as the alarm root cause analysis graph obtained in step 3, calculate the probability of k different graph structures appearing in historical data, select the structure with the highest probability as the final result of the alarm root cause analysis graph, and obtain the final root cause of this alarm.

[0068] From the hardware level, Figure 3 As shown in the figure, this manual shows a hardware structure diagram of the computer device where the root cause locating device of the fault is located. Figure 3 The computer device includes a network interface, a memory, and a storage interconnected with a processor respectively, and the memory includes a fault root cause locating device.

[0069] The contents not described in detail in the specification of the present invention belong to the prior art known to the professional and technical personnel in the field. It is pointed out here that the above description helps those skilled in the art to understand the invention, but does not limit the protection scope of the invention. Any equivalent replacement, modification and / or simplification of the above description without departing from the essence of the invention falls within the protection scope of the invention.

Claims

1. A method for discovering root causes of failures based on a storage device model, characterized in that: Using the connectivity graph, alarm root cause analysis graph, and root cause derivation graph, the norm is calculated according to the following formula: Where d is the norm, s is the number of frequent items in the connected graph, and p y i It represents the probability value of the previous fault leading to the next fault in the alarm root cause analysis diagram, p i Represents the root cause derivation diagram with p y i The probability value that the previous abnormal indicator between nodes with the same connected fault node causes another abnormal indicator; Select k alarm root cause analysis graphs with the smallest norm value, where k is a positive integer greater than 1, determine the graph structure with the same node name as the alarm root cause analysis graph from the root cause derivation graph, calculate the probability of k different graph structures appearing in the historical data, select the graph structure with the highest probability as the final result of the alarm root cause analysis graph, and obtain the final root cause of this alarm; The implementation of the root cause derivation diagram includes the following steps: Step A: Collect historical data, including system logs, performance data, and system alarm information; Step B, preprocessing the collected historical data; Step C, abstracting the preprocessed historical data; Step D, using the abstracted historical data to obtain the nodes of the root cause derivation graph according to the FP-Growth association analysis algorithm, and constructing the FP-Tree graph structure of the root cause derivation graph; Step E, calculating the probability of the previous item causing the next item between two frequent set items, and obtaining the connection direction between two nodes in the root cause deduction graph according to the probability value, until a complete root cause deduction graph is obtained; The connectivity graph is realized by following the steps below: Step 1: Obtain alarm data: When an alarm occurs on the server, obtain the server's real-time data within five minutes, including system logs, performance data, system alarm information, and server hardware topology, and perform data abstraction on the obtained alarm data; Step 2: Use the FP-Growth algorithm to obtain frequent items from all the alarm data. The frequent items obtained are the collection of root cause fault alarms and more frequent phenomenon alarms caused by the root cause. The probability that the first alarm causes the second alarm among the frequent items is calculated, and all the connected graphs obtained are saved.

2. The method for discovering root causes of failures based on a storage device model according to claim 1, characterized in that: The k=2.

3. The method for discovering root causes of failures based on a storage device model according to claim 1, characterized in that: The performance data in step A includes disk information, disk domain information, storage pool information, and host group information obtained based on the CMDB storage device model; the system alarm information includes CPU alarms, memory alarms, PCIE I / O device alarms, and power alarms obtained based on the CMDB storage device model; the system log includes client id, client ip, protocol id, and file system id obtained based on the CMDB storage device model; the disk information includes disk id, disk occupancy, and disk health status; the disk domain information includes disk domain id, disk domain occupancy, hot spare disk capacity, and disk domain health status; the storage pool information includes storage pool id, storage pool occupancy, and main memory type; the host group information includes host group id and host capacity utilization; the CPU alarm includes CPU configuration alarm, CPU voltage regulator alarm, CPU core temperature over-limit alarm, and CPU initialization alarm; the memory alarm includes memory configuration error alarm and memory temperature over-limit alarm; the PCIE The I / O device alarms include PCIe error alarms and I / O channel check error alarms reported by the BIOS system; the power supply alarms include power supply device voltage over-limit alarms.

4. The method for discovering root causes of failures based on a storage device model according to claim 1, characterized in that: The step B includes missing value processing, outlier processing, and discrete data encoding.

5. The method for discovering root causes of failures based on a storage device model according to claim 1, characterized in that: The step C includes extracting all the above data within five minutes of the alarm occurring in the historical data, and abstracting the extracted data into a combination of fault ID and fault topology location id according to the topological structure position of the alarm, and its attribute is the fault alarm time, wherein the fault topology location id is composed of the host group id, storage pool id, disk domain id and disk id. If the fault occurs in the disk domain, the fault topology location id is composed of the group id, storage pool id and disk domain id, and the remaining spaces are filled with 0s. The fault topology location id of the host group fault alarm is composed of the host group id, and the remaining positions are filled with 0s.

6. The method for discovering root causes of failures based on a storage device model according to claim 1, characterized in that: The step D includes using the FP-Growth association analysis algorithm to obtain frequent set items that appear within a 5-minute time period as nodes of the root cause analysis diagram. When alarms occur at multiple topological structure locations of the server, a corresponding number of FP-Tree diagrams are generated according to the number of locations.

7. The method for discovering root causes of failures based on a storage device model according to claim 1, characterized in that: The step E includes calculating the connection relationship of the root cause derivation graph composed of frequent set items. In the root cause derivation graph, two nodes on any edge with a direction represent the combination of the device name and the fault ID of each abnormal indicator obtained. Any edge with a direction is used to indicate the association relationship between the abnormal indicators represented by the two nodes on the edge. The probability on any edge is used to represent the probability p(a, b) that an abnormal indicator a causes another abnormal indicator b. The specific probability calculation method is as follows: Where s(a, b) indicates the number of times an alarm of an abnormal indicator a causes an alarm of another abnormal indicator b, a indicates the number of times alarm a occurs, and n indicates the total number of alarm data.

8. The method for discovering root causes of failures based on a storage device model according to claim 1, characterized in that: According to the actual situation of the server and the difference in alarm sensitivity, there may be two situations: the phenomenon alarm is earlier than the root cause alarm, and the root cause alarm is earlier than the phenomenon alarm. Here, the connection direction of the root cause deduction diagram is determined according to the specific server properties. The alarm type at the starting point is the effect relative to the alarm type at the end point, that is, the phenomenon alarm type, and the alarm type at the end point is the cause relative to the alarm type at the starting point, that is, the root cause alarm.

Citation Information

Patent Citations

  • Fault root cause positioning method and device, storage medium and equipment

    CN113098723A

  • System fault root cause positioning method and device, storage medium and electronic device

    CN113254254A