Method for realizing alarm noise reduction based on alarm time characteristics and topological node overlap ratio during alarm storm

Abstract

The invention provides a method for dividing conventional alarms and sudden alarms based on time, provides a method for monitoring the alarm storm based on the division mode, and defines a method for measuring the importance of topological nodes and reducing the noise of the alarms during the alarm storm. Based on the method, an alarm storm is compressed into a plurality of events, alarm root causes and alarm-free root causes are given, and noise reduction and analysis of massive alarms are achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to helping operation and maintenance industry personnel to detect alarm storm, reduce information pressure during alarm storm, and recommend root cause for core events of the storm, in particular, relates to a method for realizing alarm noise reduction based on alarm time characteristics and topology node coincidence degree during alarm storm. BACKGROUND

[0002] Alarm storm is a common phenomenon in operation and maintenance, which mainly manifests that a monitoring tool issues a large number of alarms in a short time. This phenomenon usually occurs in complex computer systems. As the company scale expands, more and more companies begin to build complex business support systems to meet the growing user demand, and there are multiple calling relationships in the system. A mature business system may need thousands of devices to support it. When any device fails, the failure will propagate along multiple calling relationships, causing more devices and services to malfunction. At this time, the monitoring tool will find a large number of abnormal phenomena in a short time, thus issuing a large number of alarms, forming an alarm storm. The current method usually starts from the similarity of the text to solve the problem of alarm storm, that is, those alarms with more repeated information are compressed into one, and only the compressed result is sent when sending a notification, so as to achieve the purpose of reducing the amount of alarm information. However, this method has three problems: 1. It cannot distinguish which alarms are the alarms that the alarm storm really needs to care about. Usually, there are some historical alarms in the monitoring tool itself, and the operation and maintenance team already knows the content of these alarms but cannot solve the corresponding problems due to the limitation of the operation and maintenance scene. Therefore, these historical alarms are put aside. When the storm occurs, these historical alarms will interfere with the analysis and reduce the troubleshooting speed. 2. In actual use, there are often alarms with high text similarity but no any topology calling relationship, which leads to some unrelated alarms being compressed together, causing the alarm notification to be missed and the operation and maintenance personnel to not know the corresponding situation in time. 3. The relationship between alarms based on text analysis cannot find the root cause of the alarm, which leads to slow operation and maintenance troubleshooting. SUMMARY

[0003] The present application proposes a method for realizing alarm noise reduction based on alarm time characteristics and topology node coincidence degree during alarm storm. The main purpose of the invention is to find alarm storm, filter out those historical noise alarms, and reduce the notification amount received by the operation and maintenance team based on the topology node coincidence degree and node similarity of the remaining alarms, while giving a list of possible root causes of the current storm.

[0004] To solve the above problems, the present application provides a method for realizing alarm noise reduction based on alarm time characteristics and topology node coincidence degree during alarm storm, characterized in that it comprises the following steps:

[0005] S101. Alarm Preprocessing: Alarms of the same topology nodes are compressed into an alarm sequence according to a time window to generate time series data.

[0006] S102. Sudden Incident Detection: The expected number of alarms is calculated using a weighted sliding window, and the sudden alarm and its severity are determined based on the deviation between the actual number of alarms and the expected value.

[0007] S103. Storm Detection: Counts the number of active burst alerts and triggers an alert storm when the number exceeds a dynamically calculated threshold.

[0008] S104. Root Cause Analysis: Calculate the root cause probability by combining the severity of the outbreak and the importance of topological nodes, select high-probability nodes as alarm root causes, and infer potential non-alarm root cause nodes based on topological relationships.

[0009] S105. Event Generation: By calculating the topological overlap between nodes, similar nodes are merged into events, and low overlap alarms are grouped and compressed according to their importance.

[0010] Furthermore, this application also proposes that the burst detection in S102 adopts the formula bd_current =

[0011] te_current / (weighted prediction value)-1 calculates the burst level, where the weighted prediction value is the decay weighted sum of the number of alarms in the historical window.

[0012] Furthermore, this application also proposes that in S103, storm detection uses the formula sd_current=ta_current / (weighted prediction value)-1 to calculate the storm intensity, and triggers an alarm storm when the result exceeds a preset threshold.

[0013] Furthermore, this application also proposes that the importance of the topology nodes is calculated using the formula nid_current = w_u * (proportion of upstream alarm nodes) + w_d * (proportion of downstream nodes), where w_u and w_d are configurable weights.

[0014] Furthermore, this application also proposes that the root cause probability in S104 is calculated using the formula p_current=w_bd*bd_current+w_nid*nid_current, where w_bd and w_nid are configurable weights.

[0015] Furthermore, this application also proposes that the topological overlap degree in S105 is calculated by the formula tcd_ij=w_s*(upstream node overlap degree)+w_v*(topological type vector similarity), where w_s and w_v are configurable weights.

[0016] Furthermore, this application also proposes that in S101, the inference of no alarm root cause node preferentially selects the most downstream common upstream node, and if there is no common upstream node, the upstream node with the most alarm root causes is selected.

[0017] Furthermore, this application also proposes that, in S101, for alarms without upstream nodes, sub-events are generated in groups at fixed intervals according to the importance of the nodes.

[0018] The implementation system of this application includes the following modules:

[0019] 1. Alarm Acquisition Module: Collects raw alarms from the monitoring system;

[0020] 2. Preprocessing module: Compresses alarms by time window and topology node;

[0021] 3. Burst Detection Module: Performs burst calculations and storm triggering;

[0022] 4. Topology Analysis Module: Constructs a node dependency graph and calculates importance;

[0023] 5. Root Cause Location Module: Generates lists of alarm and non-alarm root causes;

[0024] 6. Practical Compression Module: Merging Alarms Based on Topology Overlap.

[0025] 7. Visualization module: Displays root cause analysis results and practical topology.

[0026] Based on this, redundant alarms are reduced through time window compression and topology merging; weighted sliding windows automatically adapt to changes in business load; and potential root causes of alarms not being triggered (such as configuration errors) are identified.

[0027] As described above, this application proposes a method for alarm noise reduction during alarm storms based on alarm time characteristics and topology node overlap. The first objective of this invention is to provide a method for identifying alarms that truly require attention during an alarm storm based on their burst level, and to determine which nodes have higher processing priority based on the importance of topology nodes, thus solving the problem that existing technologies cannot identify key alarms during alarm storms. The second objective of this invention is to provide a method for alarm noise reduction based on topology node similarity, solving the problem that alarms with similar text but unrelated business functions are easily compressed together, leading to notification omissions. Based on these two points, this invention, based on the results of the first objective, provides the root causes and non-root causes of event alarms generated by the second objective, achieving the goal of accelerating troubleshooting by moving from monitoring alarm storms to compressing alarm storms and then to finding the root causes of faults. Attached Figure Description

[0028] Figure 1The graph shows the time series converted from alarms. There are a total of 4 alarms in the graph. The fault types of the alarms are shown in the legend. The horizontal axis is the time axis. The time here is represented by the timestamp 13. It can also be represented in the common form of 2022-10-01 10:00:00. The vertical axis is the number of alarms contained in the time window corresponding to the time point.

[0029] Figure 2 This is an example of storm warning monitoring. In the graph, the x-axis represents the number of alarms, the y-axis represents time, the red dots represent points where storm warnings occur, and the blue parts represent normal points. This graph shows the relationship between the number of alarms and time from 10:00:00 to 10:25:00 on a certain day. The orange line represents the alarm threshold calculated according to the formula. When the number of alarms exceeds the threshold, the alarm is judged as a suspected storm point. Detailed Implementation

[0030] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the specific embodiments of this application will be described in detail, clearly, and completely below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this application.

[0031] It should be noted that similar reference numerals and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0032] like Figure 1 As shown, this is the time series of alarms. The figure contains 4 alarms. The fault types of the alarms are shown in the legend. The horizontal axis is the time axis, where the time is represented by a timestamp of 13. It can also be represented in the common form of 2022-10-01 10:00:00. The vertical axis is the number of alarms contained in the time window corresponding to the time point.

[0033] Example 1: Determine whether the alarm is a sudden alarm based on the alarm information, and calculate the degree of suddenness.

[0034] Assuming the following alarm data, the numbers in the first row represent the number of time windows, for example, 1 represents the first time window. For actual data, each time window corresponds to a real time period. The end time of the previous time window is the start time of the next time window. The time span of each time window remains consistent. The start time of the first time window is specified by the user, and the end time of the last time window is the current time or a user-specified time. Besides the first row, the numbers in rows 2 and 3 represent the number of times the corresponding alarm appears in the corresponding window. For example, the number 1 in column B, row 3, means that alarm B appeared once in the 0th time window. Assume the current time window is the 5th window.

[0035] 1 2 3 4 5 Alert A 0 0 0 0 1 Alert B 1 1 1 1 1

[0036] According to the formula, let ∈ be 0.1:

[0037] bd_{current}}=\frac{te_{current}{\sum^{w-1}_{\substack{j=0\\}}\frac{wj}{w+(w-1)+...+1}*e_{{current-j}+0.1}-1

[0038] The severity level of Alarm A is:

[0039] The severity level of Alarm B is:

[0040] Assuming that an alarm with a suddenness level greater than 0 is a sudden alarm, then A is a sudden alarm with a suddenness level of 9, and B is a regular alarm.

[0041] Example 2: Determine whether an alarm storm has been triggered at the current time based on the number of alarms and the number of topology nodes within a certain period.

[0042] Assuming the following alarm and topology node data, the numbers in the first row are the same as in Example 1, representing time windows. The numbers in the second row represent the number of alarms within the corresponding time window; for example, 10 in column B of the second row means there are 10 alarms in the first time window. The numbers in the third row represent the number of topology nodes within the corresponding time window; for example, 1 in column B of the third row means there is 1 alarm in the first time window.

[0043]

[0044] According to the formula, let ∈ be 0.1:

[0045] sd_{current}=\frac{ta_{current}}{\sum^{w-1}_{\substackf{j=0\\}\frac{wj}{w+(w-1)+...+1}*a_{current-j}+0.1}-1

[0046] In the 5th time window, the storm intensity is: If a storm intensity greater than 0 and the number of topology nodes involved in the alert exceeds 5, it is considered an alert storm. Therefore, an alert storm has occurred in the current time window.

[0047] like Figure 2 The figure shows an example of alarm storm monitoring. The x-axis represents the number of alarms and the y-axis represents time. The red dots in the figure are the points where alarm storms occur, and the blue part is the normal point. The figure shows the relationship between the number of alarms and time in the range of 10:00:00 to 10:25:00 on a certain day. The orange line represents the alarm threshold calculated according to the formula. When the number of alarms exceeds the threshold, the alarm is judged as a suspected storm point.

[0048] Example 3: Calculating the importance of topology nodes corresponding to sudden alarms based on topology data

[0049] Suppose that in the following system, each node is a topology node. Let node A be the topology node to be analyzed, with upstream nodes B and C. Node B generated an alarm, and downstream nodes D, E, and F. There are a total of 8 nodes in the system.

[0050] According to the formula, let ∈ be 0.1:

[0051]

[0052] Assume weight w u and w d If both are 0.5, then the topological importance of node A is:

[0053] Example 4: Calculate the similarity of topological nodes in a sudden alarm to generate alarm clusters, and generate events based on the overlap of upstream nodes of the sudden alarm nodes.

[0054] Suppose that in the following system, nodes A and B are both database nodes, nodes C and D are both middleware nodes, node E is a service node, and node F is an application node. Assume there are four types of topology nodes: application, service, middleware, and database. Then, in this order, the topology vector for node A is [1,1,1,0], and the topology vector for node B is [1,1,1,0]. The upstream nodes that overlap between nodes A and B are E and F.

[0055] According to the formula, assume that ∈ is a negligible minimum:

[0056]

[0057] Assume weight w s and w v If both are 0.5, then the topological importance of node A is:

[0058]

[0059] Example 5: Recommend alarm-affected topology nodes as suspected root causes based on the importance of topology nodes and the suddenness of alarms.

[0060] Assuming that all nodes have alarms and the burst level is 0.5, we can calculate the node importance of each node based on the node importance example above.

[0061] According to the formula:

[0062] p current =w bd *bd current +w nid *nid current

[0063] Assume weight w bd and w nid If both are 0.5, then the root cause degree of node A is: p A =0.5*0.5 + 0.5*0.5 = 0.5

[0064] Similar to the steps above, node B is 0.5, nodes C and D are 0.54, node E is 0.585, and node F is 0.705, for a total of 6 nodes. Therefore, 1.2 alarm root causes need to be selected, which is approximately 1. Node F is the most likely alarm root cause.

[0065] Example 6: Based on the upstream nodes of the suspected root cause topology nodes, the non-alarm topology nodes of the suspected root cause are given.

[0066] In the graph-topology system, if nodes A and B have alarms while other nodes do not, then the downstream node of the overlapping nodes E and F is taken as the non-alarm root cause. Since both A and B have alarms with the same burst level and node importance, the alarm root causes are A and B.

[0067] In summary, this application proposes a method for alarm noise reduction during alarm storms based on alarm time characteristics and topology node overlap. The first objective of this invention is to provide a method for identifying alarms that truly require attention during an alarm storm based on their burst severity, and to determine which nodes have higher processing priority based on the importance of topology nodes, thus solving the problem of existing technologies being unable to identify key alarms during alarm storms. The second objective of this invention is to provide a method for alarm noise reduction based on topology node similarity, solving the problem of similar but unrelated alarms being easily compressed together, leading to notification omissions. Based on these two points, this invention, based on the results of the first objective, provides the root causes and non-root causes of event alarms resulting from the second objective, achieving the goal of accelerating troubleshooting by moving from monitoring alarm storms to compressing alarm storms and then to finding the root causes of faults.

[0068] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for alarm noise reduction based on alarm time characteristics and topology node overlap during alarm storms, characterized in that, Includes the following steps: S101. Alarm Preprocessing: Alarms of the same topology nodes are compressed into an alarm sequence according to a time window to generate time series data. S102. Sudden Incident Detection: The expected number of alarms is calculated using a weighted sliding window, and the sudden alarm and its severity are determined based on the deviation between the actual number of alarms and the expected value. S103. Storm Detection: Counts the number of active burst alerts and triggers an alert storm when the number exceeds a dynamically calculated threshold. S104. Root Cause Analysis: Calculate the root cause probability by combining the severity of the outbreak and the importance of topological nodes, select high-probability nodes as alarm root causes, and infer potential non-alarm root cause nodes based on topological relationships. S105. Event Generation: By calculating the topological overlap between nodes, similar nodes are merged into events, and low overlap alarms are grouped and compressed according to their importance.

2. The method for alarm noise reduction based on alarm time characteristics and topology node overlap during an alarm storm, as described in claim 1, is characterized in that... In S102, the burst detection uses the formula bd_current=te_current / (weighted prediction value)-1 to calculate the burst level, where the weighted prediction value is the attenuation weighted sum of the number of alarms in the historical window.

3. The method for alarm noise reduction based on alarm time characteristics and topology node overlap during an alarm storm, as described in claim 1, is characterized in that... In S103, storm detection uses the formula sd_current=ta_current / (weighted prediction value)-1 to calculate the storm intensity. When the result exceeds the preset threshold, an alarm storm is triggered.

4. The method for alarm noise reduction based on alarm time characteristics and topology node overlap during an alarm storm, as described in claim 1, is characterized in that... The importance of the topology nodes is calculated using the formula nid_current = w_u * (proportion of upstream alarm nodes) + w_d * (proportion of downstream nodes), where w_u and w_d are configurable weights.

5. The method for alarm noise reduction based on alarm time characteristics and topology node overlap during an alarm storm, as described in claim 1, is characterized in that... In S104, the root cause probability is calculated using the formula p_current=w_bd*bd_current+w_nid*nid_current, where w_bd and w_nid are configurable weights.

6. The method for alarm noise reduction based on alarm time characteristics and topology node overlap during an alarm storm, as described in claim 1, is characterized in that... In S105, the topological overlap is calculated using the formula tcd_ij=w_s*(upstream node overlap)+w_v*(topological type vector similarity), where w_s and w_v are configurable weights.

7. The method for alarm noise reduction based on alarm time characteristics and topology node overlap during an alarm storm, as described in claim 1, is characterized in that... In S101, the inference of no alarm root cause node preferentially selects the most downstream common upstream node. If there is no common upstream node, the upstream node that connects the most alarm root causes is selected.

8. A method for alarm noise reduction based on alarm time characteristics and topology node overlap during an alarm storm, as described in claim 1, characterized in that... In S101, for alarms without upstream nodes, sub-events are generated by grouping nodes at fixed intervals according to their importance.

Images