An alarm data processing method and system, electronic device, and storage medium

Abstract

This invention provides an alarm data processing method and system, electronic device, and storage medium. The method includes: classifying various custom alarm compression rules according to their respective scopes, and distributing each alarm compression rule to a corresponding computing cluster; acquiring alarm data; the alarm data includes multiple alarms to be processed; distributing each alarm to be processed to the computing cluster corresponding to the scope it belongs to for concurrent compression processing; by classifying the scopes to which the alarm compression rules belong, a domain-specific concurrent processing process is formulated to improve the matching efficiency between alarms and alarm rules, improve accuracy and timeliness, and improve the processing performance of the alarm lifecycle, thereby improving the efficiency of operation and maintenance collaboration and avoiding production failures.

Description

Technical Field

[0001] This invention belongs to the field of data processing technology, and more specifically, relates to an alarm data processing method and system, electronic device, and storage medium. Background Technology

[0002] In recent years, automated operation and maintenance (O&M) tools have received increasing attention in the industry as part of IT (Internet Technology) infrastructure. Well-known domestic and international internet companies have launched automated O&M management platforms tailored to their own development needs. Among these, a unified alarm management platform is an O&M tool designed to help enterprises effectively manage and process various alarm information. As enterprises grow in size and complexity, the quantity and types of alarm information are also increasing exponentially, posing significant challenges to O&M teams. Against this backdrop, the development of a unified alarm management platform has become crucial.

[0003] First, the development of unified alarm management platforms enables enterprises to centrally manage and monitor alarm information from multiple systems and devices. Second, these platforms make alarm processing more intelligent and automated. Traditional alarm handling often requires manual processing and judgment by operations and maintenance personnel, which is not only time-consuming and labor-intensive but also prone to human error. Modern unified alarm management platforms, by introducing intelligent alarm processing mechanisms, can automatically analyze and identify the priority and urgency of alarm information, and intelligently dispatch and allocate alarms. Furthermore, the development of unified alarm management platforms has also promoted the visualization and reporting of alarm information. Finally, the development of unified alarm management platforms has also promoted the integration and collaboration of alarm information.

[0004] Currently, existing technologies offer custom compression rule noise reduction, allowing users to set compression conditions for hosts, services, alarm content, and alarm objects according to their own needs. After the settings are complete, the alarms will be compressed and denoised according to the alarm compression rules.

[0005] However, alarm compression rules are quite flexible and diverse, which also affects the efficiency of subsequent alarm processing. For example, if alarm compression is incorrect or fails to take effect, it can lead to a large number of alarm storms, email and SMS backlogs, and production failures. Summary of the Invention

[0006] In view of this, the purpose of the present invention is to provide an alarm data processing method and system, electronic device, and storage medium, which are used to improve the matching efficiency of alarms and alarm rules, and improve accuracy and timeliness by classifying and dividing the scope to which the alarm compression rules belong.

[0007] The first aspect of this application discloses a method for processing alarm data, including:

[0008] Each custom alarm compression rule is categorized according to its scope, and each alarm compression rule is then distributed to the corresponding computing cluster; the scope includes: within the system, across systems, and within a single machine;

[0009] Acquire alarm data; the alarm data includes multiple alarms pending processing;

[0010] Based on the scope to which each of the pending alarms is hit, each pending alarm is distributed to the computing cluster corresponding to the hit scope for concurrent compression processing.

[0011] Optionally, the step of distributing each of the pending alarms to the computing cluster corresponding to the scope of the pending alarm for concurrent compression processing includes:

[0012] Determine the target alarm compression rules corresponding to each of the aforementioned alarms to be processed;

[0013] The alarms to be processed are distributed to the computing clusters where the corresponding target alarm compression rules are located for concurrent compression processing.

[0014] Optionally, the step of distributing each of the pending alarms to the computing cluster corresponding to the scope of the pending alarm for concurrent compression processing includes:

[0015] Determine whether the current pending alarms only match alarm compression rules within a single machine, across systems, or within a single system;

[0016] If the current pending alarm only matches the cross-system alarm compression rule, then the current pending alarm will be sent to the cross-system computing cluster for compression processing.

[0017] If the current pending alarm only matches the alarm compression rules in the system, then the current pending alarm will be sent to the computing cluster in the system for compression processing.

[0018] If the current pending alarm only matches the alarm compression rule within a single machine, then the current pending alarm will be sent to the computing cluster within that single machine for compression processing.

[0019] If the current pending alarm matches at least two of the alarm compression rules that match within a single machine, across systems, and within a system, then the current pending alarm is sent to the conflict job cluster for compression processing.

[0020] If the current pending alarm does not match any of the alarm compression rules within a single machine, across systems, or within a system, then the current pending alarm will be directly transferred.

[0021] Optionally, before determining whether the current alarm to be processed only matches alarm compression rules within a single machine, across systems, or within a system, the method further includes:

[0022] Determine whether the associated alarms of the currently pending alarm are being compressed;

[0023] If so, the currently pending alarm is sent to the computing cluster where the associated alarm is located for compression.

[0024] Optionally, before classifying the various custom alarm compression rules according to their respective scopes, the method further includes:

[0025] Configure the elements of each alarm compression rule, including: compression conditions, compression time window, and merging dimension.

[0026] Optionally, before distributing each of the pending alarms to the computing cluster corresponding to the scope hit by each pending alarm for concurrent compression processing, the method further includes:

[0027] Each of the aforementioned alarms to be processed is preprocessed.

[0028] Optionally, the step of classifying the various custom alarm compression rules according to their respective scopes and distributing each of the alarm compression rules to the corresponding computing clusters includes:

[0029] If the alarm compression rule merges alarms from multiple physical subsystems, then the scope of the alarm compression rule is cross-system.

[0030] If the alarm compression rule merges alarms from a single physical subsystem, then the scope of the alarm compression rule is within the system.

[0031] If the alarm compression rule merges alarms from a single host, then the scope of the alarm compression rule is within that single host.

[0032] The second aspect of this application discloses an alarm data processing system, including:

[0033] The rule classification module is used to classify each custom alarm compression rule according to its scope and distribute each alarm compression rule to the corresponding computing cluster; wherein, the scope includes: within the system, across systems, and within a single machine;

[0034] The acquisition module is used to acquire alarm data; the alarm data includes multiple alarms to be processed.

[0035] The distribution and processing module is used to distribute each of the alarms to be processed to the computing cluster corresponding to the scope of the scope to be processed for concurrent compression processing.

[0036] A third aspect of this application discloses an electronic device, comprising:

[0037] One or more processors;

[0038] A storage device on which one or more programs are stored;

[0039] When one or more programs are executed by one or more processors, the one or more processors implement the alarm data processing method as described in any one of the first aspects of this application.

[0040] The fourth aspect of this application discloses a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the alarm data processing method as described in any one of the first aspects of this application.

[0041] As can be seen from the above technical solution, the alarm data processing method provided by the present invention includes: classifying each custom alarm compression rule according to its scope, and distributing each alarm compression rule to the corresponding computing cluster; acquiring alarm data; the alarm data includes multiple alarms to be processed; according to the scope hit by each alarm to be processed, distributing each alarm to be processed to the computing cluster corresponding to the scope hit for concurrent compression processing; by classifying and dividing the scope to which the alarm compression rules belong, a domain-specific concurrent processing process is formulated to improve the matching efficiency of alarms and alarm rules, improve accuracy and timeliness, improve the processing performance of the alarm lifecycle, thereby improving the efficiency of operation and maintenance collaboration and avoiding production failures. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a flowchart of an alarm data processing method provided in an embodiment of the present invention;

[0044] Figure 2 This is a schematic diagram of the custom alarm compression rules involved in an alarm data processing method provided in an embodiment of the present invention;

[0045] Figure 3This is a schematic diagram illustrating the compression dimension definition involved in an alarm data processing method provided in an embodiment of the present invention;

[0046] Figure 4 This is a schematic diagram illustrating the division of alarm rule scope in an alarm data processing method provided by an embodiment of the present invention;

[0047] Figure 5 This is a schematic diagram of the alarm compression rule processing flow involved in an alarm data processing method provided in an embodiment of the present invention;

[0048] Figure 6 This is a flowchart illustrating the alarm compression rule algorithm judgment and processing involved in an alarm data processing method provided in an embodiment of the present invention;

[0049] Figure 7 This is a schematic diagram of an alarm data processing system provided in an embodiment of the present invention;

[0050] Figure 8 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] In this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0053] This is an explanation:

[0054] Alarm compression: When a large number of alarms are generated in a short period of time, alarm compression can be performed to merge multiple alarms into one, reducing alarm noise.

[0055] Alarm rules: These are configurations that specify an action to be performed on alarms that meet certain conditions at a defined time point or time period.

[0056] This application provides an alarm data processing method to address the problem that the definition of alarm compression rules in the prior art is relatively flexible and diverse, which also affects the processing efficiency of subsequent alarm rules. For example, if the alarm compression is incorrect or ineffective, it can lead to a large number of alarm storms, accumulation of emails and SMS messages, and other situations, causing production failures.

[0057] See Figure 1 The alarm data processing method includes:

[0058] S101. Classify each custom alarm compression rule according to its scope, and distribute each alarm compression rule to the corresponding computing cluster.

[0059] The scope includes: within the system, across systems, and within a single machine.

[0060] In previous alarm compression rule-based solutions, rules applied to all alarm objects. Only by gathering all alarm objects could the system determine which alarm objects each compression rule matched, and then compression was performed based on a time-triggered window. However, based on actual compression rule configurations in production environments, it was found that the scope of application for different alarm compression rules varies.

[0061] Some alarm compression is applied to alarm content on a single machine. For example, if a machine is still in the testing phase, all disk, memory, and CPU (Central Processing Unit) alarms generated by this machine will be compressed into a single alarm. The scope of this compression rule is limited to the single machine itself.

[0062] In addition, some alarm compression rules apply to a single system. For example, alarms within a certain system, such as alarms belonging to system A, can be compressed by the system administrator into a single alarm based on the disk usage of the same file system across different hosts. This is a file system-wide merging of alarms, but it only applies to system A. Systems B or C outside of system A may have their own independent compression logic.

[0063] In addition, within the entire alarm system, independent of all other systems, there may exist a globally effective compression rule. For example, alarms with the same key attributes can be merged into a single alarm. For instance, alarms with the same name, alarm object, and alarm IP (Internet Protocol) will be automatically compressed into one alarm, avoiding the generation of many duplicate alarms.

[0064] In summary, the scope of alarm compression rules can be divided into three categories: cross-system, intra-system, and intra-machine. This scope division improves the efficiency of parallel job execution. Furthermore, the hash algorithm for matching alarm objects within their respective scopes facilitates computation, and the reasonable distribution of data reduces memory and CPU load while also improving computational efficiency.

[0065] like Figure 4 As shown, this illustrates the distribution of alarm compression rules. After identifying the scope to which the alarm compression rule belongs, the channel through which the rule is distributed is confirmed. The alarm data stream is distributed to three scopes.

[0066] It's important to note that the alarm system targets the most critical daily on-call and fault handling scenarios in operations and maintenance. Its excellent alarm handling and correlation capabilities directly impact the accuracy of alarm handling and the efficiency of operations and maintenance. After monitoring tools trigger alarms, they send alarms one by one to the alarm system. The alarm system analyzes and handles the alarms based on their content, executing corresponding automated rules to achieve a self-handling process. For example, if an alarm is triggered by system A, it will automatically notify the operations and maintenance personnel of system A, including product managers, front-line operations personnel, second-line operations personnel, and shift supervisors, etc. This is equivalent to matching the matched alarm objects with rules for automatic SMS notifications. Another example is alarms about host downtime; upon discovery, immediate attempts should be made to repair the fault, correlating with self-healing handling and implementing automated rules for automatic machine restart, saving labor costs and improving operational efficiency.

[0067] The correctness and timeliness of matching alarm objects with alarm rules directly impact the accuracy of alarm execution and the efficiency of overall operations and maintenance. Currently, the common design involves real-time stream processing via a streaming engine like Spark or Flink after an alarm is triggered. This process matches the alarm against specific alarm rules. The relationship between alarms and alarm rules is one-to-many, meaning an alarm may match multiple rules, including notification, dispatch, compression, suppression, and self-healing rules. All rules are executed sequentially according to priority. Each alarm iterates through all rules, checking for compliance with rule conditions and executing the corresponding rule action.

[0068] A specific alarm system can have multiple functions, such as modifying alarm names and description rules, modifying alarm level rules, locating rich rules, automatically matching alarms to handlers, customizing compression rules, and alarm shutdown.

[0069] Alarm compression rules are a crucial part of alarm handling. Alarm compression merges multiple alarms within a certain timeframe that are considered to be caused by the same event into a single alarm for processing. There are primary and secondary alarms; subsequent actions taken on the primary alarm will automatically affect the secondary alarms. For example, closing the primary alarm will also close all subsequent secondary alarms. The definition of alarm compression rules is quite flexible and diverse, which also affects the efficiency of subsequent alarm handling. For instance, incorrect or ineffective alarm compression can lead to a massive alarm storm, a backlog of emails and SMS messages, and ultimately, production failures.

[0070] This application provides custom alarm compression rules and categorizes alarms according to these rules, thereby improving alarm handling efficiency.

[0071] Optionally, before categorizing the various custom alarm compression rules by their respective scopes, the system also includes configuring the elements of each alarm compression rule, including compression conditions, compression time windows, and merging dimensions.

[0072] The elements of an alarm rule include: basic information and defined conditions. Defining an alarm rule is equivalent to telling the alarm system that when an alarm that meets the defined conditions is received within a specified time frame, corresponding actions should be triggered, such as sending an SMS, email, or dispatching a work order. The definition of alarm rules is configured by the personnel responsible for system maintenance; that is, it is configured as needed based on the system's alarm situation, for example, for the personnel who need to be notified or dispatched, i.e., the administrators of each system.

[0073] Alarm rules are the rules that generate alarms, while alarm compression rules are the rules that compress alarms.

[0074] as follows Figure 2 As shown, the alarm compression rules in this application need to set several elements, including compression conditions, compression time window, and merging dimensions.

[0075] The compression conditions (defined conditions) can be: if the alarm source is equal to public cloud-transaction monitoring, the physical subsystem is equal to CCB Life, or the physical subsystem is equal to tenant-side Dragon Boat, then the compression conditions are met.

[0076] The time window can be selected from a dropdown menu in minutes; the merging dimensions can be multiple, such as transaction code type, indicators, etc.

[0077] In other words, this application adds a definition for alarm compression dimensions.

[0078] Generally, alarm compression is based on matching relevant key alarm attributes, such as whether the alarm object and alarm name match. Adding custom compression dimensions to support regular expression-based field matching or extraction might require separate adaptation, creating additional alarm attribute fields, and then adding them to the judgment conditions. This application, to support flexible alarm compression mechanisms across different scopes, adds an abstract definition of alarm dimensions. Alarm compression dimensions can be extracted from alarm fields using keywords. Defining compression dimensions is equivalent to automatically grouping and compressing alarm objects with the same key attributes, saving on defining many alarm compression conditions. (See below.) Figure 3 As shown.

[0079] In dimension management, you can configure dimension name, dimension code, source field, regular expression, source, operation, etc.

[0080] S102, Obtain alarm data.

[0081] The alarm data includes multiple pending alarms.

[0082] Specifically, the alarm data can be obtained through the unified alarm management platform. Of course, other methods of obtaining the data are also possible, but they will not be elaborated here. The appropriate method will be determined based on the actual situation, and all of them are within the scope of protection of this application.

[0083] S103. Based on the scope of each pending alarm, distribute each pending alarm to the computing cluster corresponding to the scope for concurrent compression processing.

[0084] Alarm compression rules have been systematically categorized, and their scope can be divided into global (cross-system), system-wide, and single-machine alarm compression rule scopes.

[0085] In the unified alarm management platform, after alarm data is rotated to alarm compression rules, it is processed concurrently by matching different scopes of alarm compression rules to alarm objects.

[0086] In other words, based on user-defined alarm compression rules, the scope of each rule is identified: cross-system, intra-system, or single-machine. The background computation process distributes the alarm compression rules to different computing clusters based on the judgment results. After alarm data flows in, it is processed concurrently according to the scope matching the alarm compression rule.

[0087] In other words, the technical field involved in this method is the unified alarm management platform in the IT automation operations and maintenance field. Through features such as centralized management, intelligent processing, visual reporting, and integrated collaboration, it helps enterprises better manage and process alarm information, improve operational efficiency and system stability, and provide strong support for the smooth operation of business. With the continuous advancement of technology and the evolving needs, the unified alarm management platform will continue to grow and develop, bringing more convenience and value to enterprise operations and maintenance.

[0088] The overall matching and routing algorithm for alarm compression divides the computation flow into several different branch routes for judgment and concurrent processing. First, it determines: 1) whether the alarm hits across systems; if it hits across systems, it checks if it hits within the system or on a single machine. If it hits, it flows to the conflict job; otherwise, it flows to the cross-system job. 2) if it doesn't hit across systems but hits within the system, it checks if it hits within a single machine. If it hits within a single machine, it flows to the conflict job; otherwise, it flows to the intra-system job. 3) if it doesn't hit across systems or within the system, it checks if it hits within a single machine. If it hits within a single machine, it flows to the single-machine job; otherwise, it flows directly. Alarms that don't hit any of the scopes are equivalent to alarms without defined compression rules, directly skipping the alarm compression processing stage and flowing to the next stage of alarm processing for further processing. Here, cross-system, intra-system, and intra-machine are all scopes for alarm compression rules.

[0089] Data splitting can improve the performance of parallel processing of rules in different scopes; on the other hand, it also helps to split data and avoid the accumulation of data streams.

[0090] It's important to note that operation and maintenance monitoring tools and platforms typically trigger / generate alarms automatically when fixed thresholds are configured and reached. Situations such as network outages, intermittent interruptions, system upgrades and updates, and situations involving numerous devices and a large amount of monitored content can generate massive numbers of alarms. During a failure, the resulting alarm storm can overwhelm mobile phones and emails, making it difficult for operations and maintenance personnel to sift through the massive amount of alarms and easily leading to the overlooking of important alarms. Therefore, operations and maintenance tools perform alarm compression on a unified alarm platform. Before compression, duplicate alarms are removed based on the alarm object or object ID (Identification Number). Similar alarms or alarms from the same source (different types of information from the same device) are then compressed again.

[0091] Generally, alarm compression is divided into two compression mechanisms:

[0092] The algorithm-based intelligent noise reduction works by extracting relevant keywords from alarms within a specified time period and placing them in a set of templates. The algorithm then compares the alarms, and alarms that meet the requirements are compressed, integrated, and then notified.

[0093] Customizable compression rules for noise reduction: Users can set compression conditions for hosts, services, alarm content, and alarm objects according to their own needs. After the settings are completed, the alarms will be compressed and denoised according to the alarm compression rules.

[0094] This application optimizes alarm compression and noise reduction for the second type of custom compression rule. In other words, in order to optimize the intermediate processing flow of alarm matching alarm compression rule and improve the processing efficiency during the alarm lifecycle, this alarm data processing method is proposed.

[0095] In this embodiment, each custom alarm compression rule is categorized according to its scope, and each alarm compression rule is distributed to the corresponding computing cluster. Alarm data is acquired, including multiple alarms to be processed. Based on the scope to which each alarm to be processed is located, each alarm to be processed is distributed to the computing cluster corresponding to the scope to be processed for concurrent compression. By classifying and dividing the scope to which the alarm compression rules belong, a domain-specific concurrent processing process is formulated to improve the matching efficiency between alarms and alarm rules, improve accuracy and timeliness, and improve the processing performance of the alarm lifecycle, thereby improving the efficiency of operation and maintenance collaboration and avoiding production failures.

[0096] It is worth noting that existing technology provides a multi-dimensional intelligent alarm compression method and process based on time series monitoring. The specific approach involves reading all alarm data within a set time period when an alarm is generated, performing root cause analysis to determine the correlation between alarms, and identifying the root causes of related alarms. The root cause analysis results are then marked on each alarm. Based on the results, alarms are either merged and compressed or new events are generated. Alarms that cannot be compressed based on root cause analysis are compressed according to resource type. Alarms lacking resource type information or not meeting the resource type compression criteria are compressed according to professional groups.

[0097] Existing alarm compression designs only address three compression scenarios: first, compressing related alarms identified through root cause analysis; second, compressing alarms based on resource type if the root cause analysis fails; and third, compressing alarms based on professional group if the resource type fails. This essentially fixes the three scenarios for alarm compression, matching them sequentially according to priority. Custom compression rules outside these three scenarios may not be compatible, and the accuracy of compression cannot be guaranteed.

[0098] This embodiment is applicable to custom alarm compression rules, which means that alarm compression rules can be defined in a flexible and diverse manner, and can also improve the matching efficiency between alarms and alarm rules.

[0099] Optionally, step S103 above, which distributes each pending alarm to the corresponding computing cluster for concurrent compression processing based on the scope hit by each pending alarm, includes:

[0100] Determine the target alarm compression rules corresponding to each pending alarm.

[0101] The alarms to be processed are distributed to the computing clusters where their respective target alarm compression rules are located for concurrent compression processing.

[0102] In other words, the target alarm compression rule for each alarm to be processed is determined based on the matching relationship between the alarm to be processed and the alarm compression rule.

[0103] The pending alarm is distributed to the computing cluster where the target alarm compression rule for the pending alarm is located, so that the computing cluster can compress the pending alarm according to the target alarm rule, and the various computing clusters can process it concurrently.

[0104] Optionally, step S103 above, which distributes each pending alarm to the corresponding computing cluster for concurrent compression processing based on the scope hit by each pending alarm, includes:

[0105] Determine whether the current pending alarms only match alarm compression rules within a single machine, across systems, or within a single system.

[0106] If the current pending alarm only matches the cross-system alarm compression rule, then the current pending alarm will be sent to the cross-system computing cluster for compression processing.

[0107] If the current pending alarm only matches the alarm compression rules within the system, then the current pending alarm will be sent to the computing cluster within the system for compression processing.

[0108] If the current pending alarm only matches the alarm compression rules within a single machine, then the current pending alarm will be sent to the computing cluster within that single machine for compression processing.

[0109] If the current pending alarm matches at least two of the alarm compression rules within a single machine, across systems, and within a system, then the current pending alarm will be sent to the conflict job cluster for compression processing.

[0110] If the current pending alarm does not match any of the alarm compression rules within a single machine, across systems, or within a system, the current pending alarm will be directly transferred.

[0111] Specifically, such as Figure 5As shown, after enriching, filtering, deduplicating, and adjusting for compression of alarm data, each pending alarm is routed by subsystem. Alarms that only match cross-system compression are distributed to the cross-system computing cluster (Kaflav) for compression and aggregation; alarms that only match intra-system compression are distributed to the intra-system computing cluster (Kaflav) for compression and aggregation; and alarms with conflicting subsystem compression rules are distributed to the computing cluster (Kaflav) for compression and aggregation, with the priority being cross-system compression > intra-system compression > single-machine compression. Additionally, each pending alarm is routed by IP address, and alarms that only match intra-machine compression are distributed to the intra-machine computing cluster (Kaflav) for compression and aggregation. Alarms that do not match any compression rule are directly transferred and stored in the database.

[0112] See Figure 6 The above judgment can be broken down into multiple judgments, specifically:

[0113] 1. Determine whether the alarm matches the cross-system alarm compression rule.

[0114] If the alarm matches the cross-system alarm compression rule, proceed to step 2. If the alarm does not match the cross-system alarm compression rule, proceed to step 3.

[0115] 2. Determine whether the alarm matches the alarm compression rules within the system or on a single machine.

[0116] If the alarm matches the alarm compression rules within the system or a single machine, it is transferred to the conflict job cluster; otherwise, it is transferred to the cross-system computing cluster.

[0117] After hitting one scope, it is necessary to determine whether other scopes are hit in order to judge whether there is a conflict between the scopes to which the alarm compression rule belongs.

[0118] This step can also determine whether other subsystems have rules within the same system in the cross-system alarm compression rules. If not, the alarm is distributed to the cross-system computing cluster; if so, it is distributed to the conflicting job cluster.

[0119] First, determine if the alarm matches within the system. If it does, check if the subsystem has alarm compression rules within the system. If it does, determine if it's suitable for a single machine. If it's not suitable for a single machine, distribute it to a cross-system cluster. If the alarm matches within the system, or if the subsystem has alarm compression rules within the system, or if it's suitable for a single machine, distribute it to the conflicting job cluster.

[0120] Specifically, if an alarm compression rule is defined that involves alarm compression rules between systems A, B, and C, it means that alarm data from systems A, B, or C will flow into this cross-system rule operation. However, it is still necessary to further determine whether there are any conflicts in the rules set by systems A, B, and C themselves.

[0121] If system A also sets another rule targeting subsystem A, it's equivalent to the rule operation for system A needing to be transferred to the system's internal rule execution. In this case, the scope of the ABC rule conflicts with that of the subsystem A rule. Therefore, both rules need to be placed in the fourth data stream. For conflicting rules, they are executed according to priority. If alarm data involves systems B and C, the ABC cross-system rule is executed first. If alarm data involves system A, the ABC cross-system rule is executed first, followed by the subsystem A's own rule.

[0122] 3. Determine whether the alarm matches the alarm compression rules of the system internal pressure.

[0123] If the alarm matches the alarm compression rule for system internal pressure, proceed to step 4. If the alarm does not match the alarm compression rule for system internal pressure, proceed to step 5.

[0124] 4. Determine whether the alarm matches the alarm compression rules within the single machine.

[0125] If the alarm matches the alarm compression rules within a single machine, it is transferred to the computing cluster of the conflicting job; if the alarm does not match the alarm compression rules within a single machine, it is transferred to the computing cluster within the system.

[0126] 5. Determine whether the alarm matches the alarm compression rules within the single machine.

[0127] If an alarm matches the alarm compression rules within a single machine, it is transferred to the computing cluster within that single machine; if the alarm does not match the alarm compression rules within a single machine, it is directly transferred to the next node in the unified alarm management platform.

[0128] In other words, when alarm data flows to the alarm compression rules, it is split into five streams. If it is determined that a certain alarm to be processed only matches the alarm compression rules in the system, it flows into the queue cache in the system.

[0129] The scope is divided into three types: global (cross-system), system-wide, and single-machine. However, within the specific alarm data stream, there are those with alarm compression rules and those without. Compressed data is further divided into global, system-wide, single-machine, and data where global, system, and single-machine scopes overlap, resulting in five data stream items. Data without compression rules flows directly, while data with compression rules is categorized into alarm data that only hits global scope rules, alarm data that only hits system scope rules, alarm data that only hits single-machine scope rules, and alarm data that hits a scope where single-machine, global, and system scopes overlap.

[0130] If it is determined that a pending alarm only matches the alarm compression rules within a single machine, it will flow into the queue cache within that single machine.

[0131] If an pending alarm is determined to have matched a global compression rule, it flows into the global queue cache, which is also the cross-system queue cache. The scope of this global scope is cross-system.

[0132] If a pending alarm matches compression rules across multiple scopes—for example, it matches both system-wide and single-machine rules—it flows into the fourth data cache queue, where it is handled by a separate conflict processing cluster. This cluster processes the alarms according to the priority order: global rules > system-wide rules > single-machine rules.

[0133] If a pending alarm does not match any of the alarm compression rules, the processing flow will skip the alarm compression rules and proceed to the subsequent alarm handling actions.

[0134] In previous rule designs, different rules were simply assigned different priorities, such as system-wide rules > standalone rules. All alarm objects were then grouped together for centralized processing. This essentially meant only retaining the processing cluster for the last conflicting rule mentioned in this article, assuming all rules might conflict and must be grouped together and resolved through alarm priority. However, this relatively simple and singular design structure requires a significant amount of machine memory to retain all alarm objects within the window period. Furthermore, the priority-based processing method also impacts alarm execution efficiency.

[0135] After optimizing the design process and handling scheme in this application, alarm compression rules can be concurrently distributed to different scope computing clusters for concurrent handling. Alarm objects can also be pre-distributed according to the object matching rules, thereby improving the overall handling efficiency of alarm objects.

[0136] Priority determination principle for alarm compression rules. Alarms without conflicts can be distributed in parallel to different computing clusters and processed in parallel according to the matching compression rule scope. For alarms with identified conflicts, compression processing is based on a combination of scope and time window. Alarms are merged according to the time window; if the time window first meets the rule, it is compressed first; if both time windows meet the rule, they are merged according to the rule's scope priority.

[0137] Specifically, for tasks with scope conflicts: alarms are merged according to time windows. If a rule is satisfied within a time window first, it is merged first. If both time windows are satisfied, the rules are merged according to their priority. If both time windows and priorities conflict, the rules are merged first according to their compilation order. In other words, if two rules have the same time window definition and both are rules within system A, they are considered to be double-conflicting rules and are executed according to the compilation order in which the rules are defined.

[0138] The priority of the scope is cross-system > intra-system > single-machine priority. For ease of explanation, the alarm compression rule is simplified to a rule.

[0139] In other words, the decision based on the combination of scope and time window applies when there are conflicts in the rule definition scope. Within the time window when the rule definition takes effect, the execution priority of the rules is determined, following the order of cross-system > intra-system > single-machine. If the priorities are the same, the rules are executed in the order they were compiled. If the time windows when two rules take effect are different, the priority is not only cross-system > intra-system > single-machine, but the window of the rule taking effect is considered first. If the intra-system time window takes precedence over the cross-system time window, the intra-system rule with the earlier time window is executed first. In other words, the time window is determined first, and if the time windows are the same, the rules are executed according to their priority.

[0140] In addition to alarm compression rules, this rule design structure can be extended to other alarm rules involving scope distinctions, facilitating wider application. For example, alarm dispatch rules, notification rules, and fault handling rules can also be processed separately based on their scope, improving efficiency.

[0141] In other words, the scope division and routing methods and processes, in addition to compression rules, can be extended and applied to other alarm rules involving scope division, facilitating application expansion. For example, alarm dispatch rules, notification rules, and fault handling rules can also be routed according to different scopes to improve handling efficiency.

[0142] Optionally, before determining whether the current alarm to be processed only matches alarm compression rules within a single machine, across systems, or within a system, the following steps are also included:

[0143] Determine whether the associated alarms of the currently pending alarm are being compressed.

[0144] If so, the current pending alarm will be sent to the computing cluster where the associated alarm is located.

[0145] Optionally, before distributing each pending alarm to the corresponding computing cluster for concurrent compression processing based on the scope hit by each pending alarm, the following steps are also included:

[0146] Preprocess each pending alarm.

[0147] During the alarm rule processing, some pre-processing rules can be concurrently processed by the stream computing engine. These include alarm filtering rules, alarm enrichment rules, alarm deduplication rules, and rules on whether to compress alarms.

[0148] The preprocessing methods will not be elaborated here, but can be determined according to the actual situation, and all are within the scope of protection of this application.

[0149] Optionally, step S101 above, which involves classifying each custom alarm compression rule according to its scope and distributing each alarm compression rule to the corresponding computing cluster, includes:

[0150] If an alarm compression rule merges alarms from multiple physical subsystems, then the scope of the alarm compression rule is cross-system.

[0151] If an alarm compression rule merges alarms from a single physical subsystem, then the scope of the alarm compression rule is within the system.

[0152] If an alarm compression rule merges alarms from a single host, then the scope of the alarm compression rule is within that single host.

[0153] Specifically, the alarm compression rules and their judgment conditions for each scope pair are shown in Table 1:

[0154] Table 1: Rules and conditions for scope compression

[0155]

[0156] The judgment condition is the basis for the backend to classify and distribute the compression rules defined by the user in the interface. For example, if the compression rule defined by the user contains multiple IPs, the backend determines that these multiple IPs belong to multiple systems and meet the judgment condition of multiple systems. Then the rule is classified as cross-system (global).

[0157] In other words, the specific method for determining the scope of an alarm compression rule depends on the conditions defined in the alarm rule. This article describes cross-system, intra-system, and single-machine judgment methods. The rule classification is automatically performed by the system based on the set conditions, routing the alarm to different scopes for handling.

[0158] If a rule condition is not equal to or does not contain a system, has no subsystem, contains multiple subsystems, multiple IPs belong to multiple subsystems, or contains / does not contain / is not equal to an IP (regular expression), it is judged as a cross-system scope.

[0159] If a rule condition is equal to or contains a subsystem, or if a single IP or multiple IPs enrich a subsystem, it is judged as an intra-system scope.

[0160] If the rule conditions only include alarms within a single host or IP range, it is judged as a single-machine scope.

[0161] Scope division enables hash matching within individual scopes, facilitating computation and preventing data from being stored together. In this paper, hash routing within a single machine is divided according to IP address zones, while hash routing across systems and within systems is divided according to the subsystem dimension.

[0162] An example of a cross-system alarm compression rule is as follows: The alarm rule is set to compress all alarms with alarm level = recovery status.

[0163] An example of alarm compression rules within the system is: compress all alarms with error code = xxx within a certain subsystem A.

[0164] An example of an alarm compression rule within a single machine is as follows: For host A, all alarms on the host with IP=1.1.1.1 are compressed together.

[0165] In this embodiment, the matching and processing efficiency of alarm compression rules can be improved, and the alarm processing flow can be enhanced; the scope division criteria of alarm rules are proposed, which can more accurately hit the alarm rules and improve the processing efficiency.

[0166] Another embodiment of this application provides an alarm data processing system.

[0167] See Figure 7 The alarm data processing system includes:

[0168] The rule classification module 101 is used to classify each custom alarm compression rule according to its scope and distribute each alarm compression rule to the corresponding computing cluster.

[0169] The scope includes: within the system, across systems, and within a single machine.

[0170] The acquisition module 102 is used to acquire alarm data; the alarm data includes multiple alarms to be processed.

[0171] The distribution and processing module 103 is used to distribute each pending alarm to the computing cluster corresponding to the scope of the pending alarm for concurrent compression processing.

[0172] For details on the specific working process and principles of each of the above modules, please refer to the alarm data processing method provided in the above embodiments. They will not be repeated here, and can be determined according to the actual situation. All of them are within the protection scope of this application.

[0173] In this embodiment, the rule classification module 101 classifies each custom alarm compression rule according to its scope and distributes each alarm compression rule to the corresponding computing cluster. The scope includes: within the system, across systems, and within a single machine. The acquisition module 102 acquires alarm data, which includes multiple alarms to be processed. The distribution and processing module 103 distributes each alarm to be processed to the computing cluster corresponding to the scope it hits for concurrent compression processing. By classifying the scopes to which the alarm compression rules belong, a domain-specific concurrent processing process is established to improve the matching efficiency between alarms and alarm rules, improve accuracy and timeliness, and improve the processing performance of the alarm lifecycle, thereby improving the efficiency of operation and maintenance collaboration and avoiding production failures.

[0174] Another embodiment of this application provides a storage medium on which a computer program is stored, wherein when the computer program is executed by a processor, it implements an alarm data processing method as described in any of the above embodiments.

[0175] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0176] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0177] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0178] Another embodiment of the present invention provides an electronic device, such as... Figure 8 As shown, it includes:

[0179] One or more processors 201.

[0180] Storage device 202, on which one or more programs are stored.

[0181] When one or more programs are executed by one or more processors 201, the one or more processors 201 implement the alarm data processing method as described in any of the above embodiments.

[0182] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts.

[0183] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

[0184] While several specific implementation details are included in the foregoing discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0185] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0186] The features described in the various embodiments of this specification can be substituted for or combined with each other. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0187] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0188] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method of handling alarm data, characterized by, include: Each custom alarm compression rule is categorized according to its scope, and each alarm compression rule is then distributed to the corresponding computing cluster; the scope includes: within the system, across systems, and within a single machine; Acquire alarm data; the alarm data includes multiple alarms pending processing; Based on the scope to which each of the pending alarms is hit, each of the pending alarms is distributed to the computing cluster corresponding to the scope to be hit for concurrent compression processing. The step of distributing each of the pending alarms to the corresponding computing clusters for concurrent compression processing based on the scope to which each pending alarm falls includes: Determine whether the current pending alarms only match alarm compression rules within a single machine, across systems, or within a single system; If the current pending alarm only matches the cross-system alarm compression rule, then the current pending alarm will be sent to the cross-system computing cluster for compression processing. If the current pending alarm only matches the alarm compression rules in the system, then the current pending alarm will be sent to the computing cluster in the system for compression processing. If the current pending alarm only matches the alarm compression rule within a single machine, then the current pending alarm will be sent to the computing cluster within that single machine for compression processing. If the current pending alarm matches at least two of the alarm compression rules that match within a single machine, across systems, and within a system, then the current pending alarm will be sent to the conflict job cluster for compression processing. If the current pending alarm does not match any of the alarm compression rules within a single machine, across systems, or within a system, then the current pending alarm will be directly transferred.

2. The alert data handling method of claim 1, wherein, The step of distributing each pending alarm to the corresponding computing cluster for concurrent compression processing based on the scope to which it is hit includes: Determine the target alarm compression rules corresponding to each of the aforementioned alarms to be processed; The alarms to be processed are distributed to the computing clusters where the corresponding target alarm compression rules are located for concurrent compression processing.

3. The alert data handling method of claim 1, wherein, Before determining whether the current alarm to be processed only matches alarm compression rules within a single machine, across systems, or within a system, the following steps are also included: Determine whether the associated alarms of the currently pending alarm are being compressed; If so, the currently pending alarm is sent to the computing cluster where the associated alarm is located for compression.

4. The alarm data processing method according to claim 1, characterized in that, Before classifying the various custom alarm compression rules according to their respective scopes, the following steps are also included: Configure the elements of each alarm compression rule, including: compression conditions, compression time window, and merging dimension.

5. The alarm data processing method according to claim 1, characterized in that, Before distributing each of the pending alarms to the computing cluster corresponding to the scope they hit for concurrent compression processing, the method further includes: Each of the aforementioned alarms to be processed is preprocessed.

6. The alarm data processing method according to claim 1, characterized in that, The step of classifying the various custom alarm compression rules according to their respective scopes and distributing each of the alarm compression rules to the corresponding computing clusters includes: If the alarm compression rule merges alarms from multiple physical subsystems, then the scope of the alarm compression rule is cross-system. If the alarm compression rule merges alarms from a single physical subsystem, then the scope of the alarm compression rule is within the system. If the alarm compression rule merges alarms from a single host, then the scope of the alarm compression rule is within that single host.

7. An alarm data processing system, characterized in that, include: The rule classification module is used to classify each custom alarm compression rule according to its scope and distribute each alarm compression rule to the corresponding computing cluster; wherein, the scope includes: within the system, across systems, and within a single machine; The acquisition module is used to acquire alarm data; the alarm data includes multiple alarms to be processed. The distribution and processing module is used to distribute each of the pending alarms to the computing clusters corresponding to the scopes they fall under for concurrent compression processing; wherein, the step of distributing each of the pending alarms to the computing clusters corresponding to the scopes they fall under for concurrent compression processing includes: Determine whether the current pending alarms only match alarm compression rules within a single machine, across systems, or within a single system; If the current pending alarm only matches the cross-system alarm compression rule, then the current pending alarm will be sent to the cross-system computing cluster for compression processing. If the current pending alarm only matches the alarm compression rules in the system, then the current pending alarm will be sent to the computing cluster in the system for compression processing. If the current pending alarm only matches the alarm compression rule within a single machine, then the current pending alarm will be sent to the computing cluster within that single machine for compression processing. If the current pending alarm matches at least two of the alarm compression rules that match within a single machine, across systems, and within a system, then the current pending alarm will be sent to the conflict job cluster for compression processing. If the current pending alarm does not match any of the alarm compression rules within a single machine, across systems, or within a system, then the current pending alarm will be directly transferred.

8. An electronic device, characterized in that, include: One or more processors; A storage device on which one or more programs are stored; When one or more programs are executed by one or more processors, the one or more processors implement the alarm data processing method as described in any one of claims 1-6.

9. A storage medium, characterized in that, It stores a computer program, wherein when the computer program is executed by a processor, it implements the alarm data processing method as described in any one of claims 1-6.

Images