Alarm processing method and device, computer equipment and storage medium

Abstract

The embodiment of the invention relates to the technical field of alarm processing, and discloses an alarm processing method, which comprises the following steps of: acquiring an original alarm message of each application system through each alarm monitoring tool; receiving an original alarm message through the data pipeline cluster, and carrying out current limiting and rectification processing on the original alarm message to obtain a plurality of processed target alarm messages; sending the plurality of processed target alarm messages to corresponding target message queues according to corresponding alarm levels through a data pipeline cluster; the different message queues comprise a high-level message queue, a middle-level message queue and a low-level message queue; the alarm level is determined according to index data of a monitoring index corresponding to the target alarm message; the target message queue transmits the target alarm message to a preset preprocessing module; the preprocessing module obtains the associated information from the configuration management database, and enriches the target alarm message to obtain the target alarm message after information enrichment; the preprocessing module performs multi-dimensional alarm aggregation compression on the target alarm message after information enrichment to obtain a compressed multi-dimensional alarm message; and sending the compressed multi-dimensional alarm message to a centralized alarm processing platform, so that the centralized alarm processing platform performs centralized processing according to the alarm level. According to the embodiment of the invention, the processing efficiency of alarm processing is improved, and the alarm storm can be effectively resisted.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of alarm processing, and in particular to an alarm processing method, an alarm processing device, a computer device, and a computer-readable storage medium. Background Art

[0002] At present, in the monitoring and alarm system in the IT operation and maintenance field, in order to monitor the operating status of the business system, there are multiple monitoring items in each business system. These monitoring items are monitored by different alarm monitoring tools. Each monitoring item will set a reasonable threshold range. The normal threshold range monitoring item will not generate an alarm. When the indicator data of the monitoring item exceeds the normal range, an alarm will be generated. The enterprise usually uses a centralized alarm platform to help operation and maintenance personnel to uniformly receive and centrally process alarms, thereby improving the efficiency of alarm handling and management.

[0003] When a system failure or performance problem occurs, when the alarm threshold is set too strictly or the monitoring rules are too loose, or when an abnormality occurs in the associated basic equipment (network equipment, storage equipment), the business system may suddenly trigger a large number of alarms at the level of tens of thousands, hundreds of thousands, or even millions, forming an alarm storm.

[0004] As the number of internal enterprise operation and maintenance application systems increases, the number and types of alarms become more complex, and the possibility of alarm storms is increasing. Alarm storms can cause information overload, response delays, program crashes, and other problems in the entire monitoring and alarm system, making it difficult for operation and maintenance personnel to quickly locate problems in a sea of ​​information, and greatly reducing the alarm processing efficiency of operation and maintenance personnel, causing operation and maintenance risks.

[0005] Therefore, resisting alarm storms is imperative and is a severe challenge that must be faced in the IT operation and maintenance field. Summary of the invention

[0006] In view of the above problems, an embodiment of the present invention provides an alarm processing method, an alarm processing device, a computer device and a computer-readable storage medium, which are used to solve the technical problems existing in the prior art that alarm storms are easily triggered and important alarms after the alarm storm cannot be processed in time by operation and maintenance personnel.

[0007] According to one aspect of an embodiment of the present invention, a method for processing an alarm is provided, the method comprising:

[0008] Collect original alarm messages from each application system through various preset alarm monitoring tools;

[0009] The original alarm message is received through a data pipeline cluster, and current limiting and rectification processing is performed on the original alarm message to obtain a plurality of processed target alarm messages;

[0010] Sending the processed multiple target alarm messages to corresponding target message queues according to corresponding alarm levels through the data pipeline cluster; the target message queue is at least one of multiple different message queues set according to the alarm level; the different message queues include high-level message queues, medium-level message queues and low-level message queues; the alarm level is determined according to the indicator data of the monitoring indicator corresponding to the target alarm message;

[0011] The target message queue transmits the target alarm message to a preset preprocessing module;

[0012] The preprocessing module obtains the associated information from the configuration management database, and enriches the target warning message to obtain the target warning message after enrichment;

[0013] The preprocessing module performs multi-dimensional alarm aggregation compression on the target alarm message after information enrichment to obtain a compressed multi-dimensional alarm message;

[0014] The compressed multi-dimensional alarm message is sent to a centralized alarm processing platform so that the centralized alarm processing platform performs centralized processing according to the alarm level.

[0015] In an optional manner, before receiving the original alarm message through the data pipeline cluster and performing flow limiting and rectification processing on the original alarm message to obtain a plurality of processed target alarm messages, the method includes:

[0016] Determine the data type of each alarm monitoring tool;

[0017] According to the data type, multiple data access plug-ins are selected to establish data transmission protocols with each of the target alarm monitoring tools; the multiple data access plug-ins include at least one or more of HTTP plug-ins, SNMP plug-ins and JDBC plug-ins;

[0018] The receiving the original alarm message through the data pipeline cluster, and performing current limiting and rectification processing on the original alarm message to obtain a plurality of processed target alarm messages, including:

[0019] The original alarm message is received from each of the target alarm monitoring tools according to the data transmission protocol.

[0020] In an optional manner, the receiving the original alarm message through the data pipeline cluster, and performing flow limiting and rectification processing on the original alarm message to obtain a plurality of processed target alarm messages further includes:

[0021] The data pipeline cluster sets a threshold value of the number of the original alarm messages received within a preset time;

[0022] Receiving the original warning message within the quantity threshold within the preset time;

[0023] When the quantity threshold is exceeded within the preset time, the original alarm message that exceeds the threshold is discarded or the log is printed without entering the subsequent processing flow.

[0024] In an optional manner, the receiving the original alarm message through the data pipeline cluster, and performing flow limiting and rectification processing on the original alarm message to obtain a plurality of processed target alarm messages further includes:

[0025] Extracting target field data from the field information of the original alarm message;

[0026] Determine attribute information of each of the original alarm messages according to the target field data, the attribute information including alarm title, alarm content, alarm IP address, alarm occurrence time, alarm level, alarm type and monitoring tool information;

[0027] According to the attribute information, the target attribute information of the target alarm message is obtained by adjusting the attribute format.

[0028] In an optional manner, the preprocessing module includes a high-level alarm consumer, a medium-level alarm consumer, and a low-level alarm consumer; the preprocessing module obtains related information from a configuration management database, enriches the target alarm message, and obtains the target alarm message after enrichment, including:

[0029] The target level alarm consumer obtains the target association information corresponding to the target alarm information from the configuration management database according to the IP information of the target alarm message; the target level alarm consumer is one of the high level alarm consumer, the medium level alarm consumer and the low level alarm consumer; the target association information includes one or more of the computer room information, the application system classification, the application system, the component information, the cluster information and the relevant person in charge information;

[0030] The target warning message is enriched according to the target association information to obtain an information-enriched target warning message.

[0031] In an optional manner, the preprocessing module performs multi-dimensional alarm aggregation compression on the information-enriched target alarm message to obtain a compressed multi-dimensional alarm message, including:

[0032] For the target alarm messages with enriched information of different alarm levels, multiple target alarm messages with enriched information of the same component under the same application system monitored by the same monitoring alarm tool are aggregated to obtain aggregated alarm messages based on system dimensions and alarm tool dimensions at each alarm level;

[0033] For the target alarm messages with enriched information of different alarm levels, multiple target alarm messages with enriched information of the same IP address and alarm title within the same time period are aggregated to obtain aggregated alarm messages based on time windows and alarm attributes at each alarm level;

[0034] The aggregated alarm messages based on the system dimension and the alarm tool dimension at each alarm level and the aggregated alarm messages based on the time window and the alarm attribute at each alarm level are used as compressed multi-dimensional alarm messages.

[0035] In an optional manner, for the target alarm messages with enriched information of different alarm levels, a plurality of target alarm messages with enriched information of the same application system component monitored by the same monitoring alarm tool are respectively aggregated to obtain aggregated alarm messages based on system dimensions and alarm tool dimensions at each alarm level, including: for the target alarm messages with enriched information of different alarm levels, according to different alarm compression strategies, a plurality of target alarm messages with enriched information of the same application system component monitored by the same monitoring alarm tool are respectively aggregated to obtain aggregated alarm messages based on system dimensions and alarm tool dimensions at each alarm level;

[0036] For the information-enriched target alarm messages of different alarm levels, the multiple information-enriched target alarm messages with the same IP address and alarm title within the same time period are aggregated respectively to obtain the aggregated alarm message based on the time window and alarm attribute at each alarm level, including: for the information-enriched target alarm messages of different alarm levels, according to different alarm compression strategies, the multiple information-enriched target alarm messages with the same IP address and alarm title within the same time period are aggregated respectively to obtain the aggregated alarm message based on the time window and alarm attribute at each alarm level.

[0037] According to another aspect of an embodiment of the present invention, there is provided an alarm processing device, including:

[0038] Multiple preset alarm monitoring tools are used to collect original alarm messages from various application systems;

[0039] A data pipeline cluster is used to receive the original alarm message, and perform current limiting and rectification processing on the original alarm message to obtain a plurality of processed target alarm messages;

[0040] The data pipeline cluster is further used to send the processed multiple target alarm messages to corresponding target message queues according to corresponding alarm levels; the target message queue is at least one of multiple different message queues set according to the alarm level; the different message queues include a high-level message queue, a medium-level message queue and a low-level message queue;

[0041] The target message queue is used to transmit the target alarm message to a preset pre-processing module;

[0042] The preprocessing module is used to obtain relevant information from the configuration management database, enrich the target warning message, and obtain the target warning message after enrichment;

[0043] The preprocessing module is further used to perform multi-dimensional alarm aggregation compression on the target alarm message after information enrichment to obtain a compressed multi-dimensional alarm message;

[0044] The pre-processing module is also used to send the compressed multi-dimensional alarm message to the centralized alarm processing platform, so that the centralized alarm processing platform performs centralized processing according to the alarm level.

[0045] According to another aspect of an embodiment of the present invention, there is provided a computer device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;

[0046] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the operation of the alarm processing method.

[0047] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, wherein the storage medium stores at least one executable instruction, and when the executable instruction is executed on a computer device, the computer device executes the operation of the alarm processing method.

[0048] The embodiment of the present invention collects the original alarm messages of each application system through each preset alarm monitoring tool, receives the original alarm message through the data pipeline cluster, and performs current limiting and rectification processing on the original alarm message to obtain multiple processed target alarm messages, and sends the processed multiple target alarm messages to the corresponding target message queue according to the corresponding alarm level. Among them, the target message queue is at least one of multiple different message queues set according to the alarm level, and the different message queues include high-level message queues, medium-level message queues and low-level message queues. The alarm level is determined according to the indicator data of the monitoring indicator corresponding to the target alarm message. The target message queue transmits the target alarm message to a preset preprocessing module, and the preprocessing module obtains the associated information according to the configuration management database, enriches the target alarm message, and obtains the target alarm message after information enrichment. The preprocessing module performs multi-dimensional alarm aggregation compression on the target alarm message after information enrichment to obtain the compressed multi-dimensional alarm message, and sends the compressed multi-dimensional alarm message to the centralized alarm processing platform, so that the centralized alarm processing platform performs centralized processing according to the alarm level. The embodiment of the present invention rectifies the alarm data through the data pipeline cluster, and consumes the alarm data separately in the queue, so that the alarm message can be decoupled and the stability of alarm processing can be improved; by setting up graded alarms and using different strategies to process alarms of different levels, high-level alarms are given priority, ensuring that high-level system alarms can be responded to first. Furthermore, the present application significantly reduces the number of alarms and reduces the risk of information overload by setting up multiple compression aggregation strategies.

[0049] The above description is only an overview of the technical solution of the embodiment of the present invention. In order to more clearly understand the technical means of the embodiment of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiment of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present invention. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings. In the accompanying drawings:

[0051] Figure 1 A schematic diagram showing the structure of an alarm processing device provided by an embodiment of the present invention is shown;

[0052] Figure 2 A schematic diagram showing the structure of another specific embodiment of the alarm processing device provided by an embodiment of the present invention is shown;

[0053] Figure 3It shows a schematic diagram of the structure of a data pipeline cluster in another specific embodiment of the alarm processing device provided by an embodiment of the present invention;

[0054] Figure 4 It shows a structural schematic diagram of a pre-processing module in another specific embodiment of the alarm processing device provided by an embodiment of the present invention;

[0055] Figure 5 A schematic diagram showing an aggregation method in another specific embodiment of the alarm processing device provided in an embodiment of the present invention is shown;

[0056] Figure 6 A schematic diagram showing a flow chart of an alarm processing method provided by an embodiment of the present invention;

[0057] Figure 7 A schematic diagram of the structure of a computer device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0058] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0059] Figure 1 The structure diagram of the alarm processing device provided by the embodiment of the present invention is shown, wherein the alarm processing device includes a plurality of preset alarm monitoring tools 210 , a data pipeline cluster 220 , a target message queue 230 and a pre-processing module 240 .

[0060] Among them, multiple preset alarm monitoring tools 210 are used to collect original alarm messages of each application system. Specifically, for multiple business systems that need to be monitored, there are multiple different monitoring items inside each business system. Therefore, these monitoring items are monitored by different alarm monitoring tools 210 respectively. Since different business systems need to monitor basic resources, host status, program process, log, business operation status, etc., different alarm monitoring tools 210 are configured to monitor each business system respectively, so as to collect and obtain the original alarm messages of each application system. Among them, multiple preset alarm monitoring tools 210 include zabbix, spark, prometheus, dial-up testing platform, etc. Among them, Zabbix is ​​an enterprise-level open source distributed monitoring solution, which can monitor various network parameters, server health status, application performance and many other indicators. Spark is a fast and general large-scale data processing engine, which provides efficient distributed data set operations and supports multiple programming languages, such as Python, Java, Scala, etc. For example, when processing massive log data, Spark can quickly read, clean and analyze these data. Prometheus is an open source system monitoring and alerting toolkit with efficient data collection and flexible query language for monitoring in cloud-native environments. Dial-up testing is a method of detecting the availability and performance of network services or applications by simulating user behavior. The original alarm message includes the original alarm title field, the original alarm IP field, the original alarm level field, the application system information field, etc.

[0061] The data pipeline cluster 220 is used to receive the original alarm message, and perform flow limiting and rectification processing on the original alarm message to obtain a plurality of processed target alarm messages.

[0062] Among them, since different alarm monitoring tools 210 come from different manufacturers, such as alarm monitoring tools that come with the application system, alarm monitoring tools built by the enterprise, or monitoring alarm tools from other sources. Therefore, various types of alarm monitoring tools may have different data types, data formats, and data transmission methods. Therefore, in order to achieve unified reception of alarm information generated by various alarm monitoring tools 210, the embodiment of the present invention sets a data pipeline cluster 220, and establishes a variety of data access plug-ins in the data pipeline cluster 220, thereby establishing a data transmission protocol with each of the target alarm monitoring tools 210. Specifically, the data type of each alarm monitoring tool 210 is determined; according to the data type, a variety of data access plug-ins are selected to establish a data transmission protocol with each of the target alarm monitoring tools 210; the multiple data access plug-ins include at least one or more of an HTTP plug-in, an SNMP plug-in, and a JDBC plug-in. Among them, in an embodiment of the present invention, if Figure 3 As shown, the logstash open source component is used as the data pipeline cluster 220, and multiple data access plug-ins are established in the logstash open source component. By establishing an HTTP plug-in, it supports accessing the original alarm message through HTTP request; by establishing a JDBC plug-in, the original alarm message can be accessed from the database; by establishing an SNMP plug-in, it supports accessing the original alarm message through the SNMP protocol. After establishing the data access plug-in, the logstash open source component receives the original alarm message from each of the target alarm monitoring tools according to the data transmission protocol.

[0063] In an embodiment of the present invention, in order to avoid the impact of excessive alarm storm data volume, the data pipeline cluster 220 also limits the access of the original alarm message. Specifically, the data pipeline cluster sets a threshold value for the number of original alarm messages received within a preset time; receives the original alarm message within the threshold value within the preset time; when the threshold value is exceeded within the preset time, the original alarm message is discarded or a log is printed. Among them, the logstash open source component of the embodiment of the present invention uses a throttle current limiting plug-in to limit the flow of the original alarm message. Specifically, the number threshold of the original alarm message allowed to pass within the set time period is specified, and the preset time is used as a cycle to count and limit the number of corresponding original alarm messages, and the values ​​of the two fields ipAddress (IP address field in the original alarm message) and eventName (name field of the original alarm message) in the original alarm message are extracted, and combined to form a unique identifier, so that the original alarm messages of different sources (different IPs) and different types (different event names) can be respectively limited and counted and controlled.

[0064] Among them, the data pipeline cluster 220 also rectifies the access of the original alarm message, specifically, extracts the target field data from the field information of the original alarm message; determines the attribute information of each of the original alarm messages according to the target field data, and the attribute information includes the alarm title, alarm content, alarm IP address, alarm occurrence time, alarm level, alarm type and monitoring tool information; adjusts the attribute format according to the attribute information to obtain the target attribute information of the target alarm message. Since the original alarm message includes information such as the original alarm title field, the original alarm IP field, the original alarm level field, and the application system information field, and the original alarm messages from different sources have different alarm message formats, the embodiment of the present invention extracts field information from the field information of the original alarm message, and then adjusts it according to the set attribute format, so as to obtain a target alarm message with a unified attribute format, thereby realizing the rectification of the original alarm message. Among them, the attribute format can be unified according to the following Table 1:

[0065] Table 1:

[0066]

[0067] Among them, during the rectification process, the alarm level of the original alarm message is determined according to the indicator value corresponding to each monitoring indicator of each original alarm message. For example, when the original alarm message is the CPU occupancy rate, its monitoring indicator is "CPU occupancy rate" and the indicator value is "95%". When the indicator value of the CPU occupancy rate exceeds 90%, its alarm level is determined to be extremely high, when the indicator value of the CPU occupancy rate is between 70-90%, its alarm level is determined to be high, when the indicator value of the CPU occupancy rate is between 50-70%, its alarm level is determined to be medium, and when the indicator value of the CPU occupancy rate is less than 50%, its alarm level is determined to be low.

[0068] The data pipeline cluster 220 is also used to send the processed multiple target alarm messages to the corresponding target message queue 230 according to the corresponding alarm levels.

[0069] The target message queue 230 is used to transmit the target alarm message to the preset preprocessing module 240. Among them, the target message queue 230 is at least one of a plurality of different message queues set according to the alarm level; the different message queues include high-level message queues, medium-level message queues and low-level message queues. In an embodiment of the present invention, a kafka queue can be used as a message queue. Specifically, in order to eliminate peak decoupling, the embodiment of the present invention puts the target alarm messages after rectification into different message queues respectively, and consumes them separately in different message queues.

[0070] The preprocessing module 240 is used to obtain the relevant information in the configuration management database, enrich the target alarm message, and obtain the target alarm message after information enrichment. The preprocessing module 240 includes a high-level alarm consumer, a medium-level alarm consumer, and a low-level alarm consumer. Each level of the preprocessing module corresponds to an alarm message level, so that the target alarm message can be consumed in different levels. Specifically, Figure 4As shown, taking the medium-level alarm consumer as an example, the target-level alarm consumer obtains the target association information corresponding to the target alarm information from the configuration management database according to the IP information of the target alarm message; the target-level alarm consumer is one of the high-level alarm consumer, the medium-level alarm consumer and the low-level alarm consumer. The target association information includes one or more of the computer room information, the application system classification, the application system, component information, cluster information and relevant person in charge information. The target alarm message is enriched according to the target association information to obtain the target alarm message after information enrichment. Among them, the configuration management database stores the relevant information of each application system. The alarm enrichment logic of the pre-processing module 240 associates and matches the alarm information with the system components and other information provided by the CMDB (configuration management database). For example, for a target alarm message, its alarm content is "a certain host network connection status is abnormal", its alarm monitoring tool 210 is "zabbix", the IP address "ipAddress" is: "192.162.1.101", and the alarm level is: "1, alarm". The alarm event "eventTime" is: "2024-10-01T12:00:00.000+08:00", "eventName": "The alarm title is: a certain host network connection is abnormally interrupted", and the alarm level "severity" is: "2 high level". Get the associated information of the IP from the CMDB configuration management database, including the information of the computer room, system classification, system, component, cluster, and relevant person in charge, as follows:"ipAddress":"192.162.1.101","appSys":"System","appSysShort":"HQ","bizSys":"System Type","clusterName":"Cluster A","component":"Component","componentShort":"DTF","dbOwnerA":"Database A","dbOwnerB":"Database B","depart":"Department","dutyAdmin":"Duty Officer Information","hostName":"Host Name","idc":"Computer Room","midOwnerA":"Middleware A","midOwnerB":"Middleware B","opsOwnerA":"Operation and Maintenance A","opsOwnerB":"Operation and Maintenance B".After the alarm is enriched, the information is fully described. The operation and maintenance personnel claim the alarm according to the position of the person in charge, and can judge which system has the problem through the information. After the enrichment is completed, the alarm information is as follows: "serial":"EVE20241001000001", "summary":"Alarm content: The network connection status of a certain host is abnormal", "agent":"zabbix", "ipAddress":"192.162.1.101", "eventTime":"2024-10 -01T12:00:00.000+08:00","eventName":"Alarm title: A host network connection interruption abnormality was found","severity":"2","suppressEscl":0, / / Alarm category 0: Normal alarm 1: Silent alarm "tally":1, / / Number of occurrences "appSys":"System","appSysShort":"HQ","bizSys":"System type","clusterName":"Cluster A". "component":"XXX component","componentShort":"XXX","dbOwnerA":"Database A post","dbOwnerB":"Database B post","depart":"Department","dutyAdmin":"Duty officer information","hostName":"Host name","idc":"Computer room","midOwnerA":"Middleware A post","midOwnerB":"Middleware B post","opsOwnerA":"Operation and maintenance A post","opsOwnerB":"Operation and maintenance B post". It can be seen that the embodiment of the present invention sets a buffer before pushing the alarm to the centralized alarm processing platform, and enriches the alarm message with information, so that the alarm can be processed more accurately.

[0071] In the embodiment of the present invention, after the alarm message is enriched with information, the alarm message can be dynamically graded and classified again according to the actual situation of the alarm message, and the alarm level can be adjusted to make the classification of the alarm message more reasonable.

[0072] The preprocessing module 240 is also used to perform multi-dimensional alarm aggregation and compression on the target alarm message after information enrichment to obtain a compressed multi-dimensional alarm message. The preprocessing module 240 of the embodiment of the present invention aggregates related alarm information together to remove duplication, reduce the number of alarms and improve processing efficiency. Among them, for the target alarm messages after information enrichment of different alarm levels, according to different alarm compression strategies, multiple target alarm messages after information enrichment of the same component under the same application system monitored by the same monitoring alarm tool are respectively aggregated to obtain aggregated alarm messages based on the system dimension and the alarm tool dimension of the respective alarm messages at each alarm level. Among them, for the target alarm messages after information enrichment of different alarm levels, the target alarm messages after information enrichment of different alarm levels can be compressed according to different alarm compression strategies. In the embodiment of the present invention, multiple target alarm messages after information enrichment with the same IP address and alarm title in the same time period can also be aggregated to obtain aggregated alarm messages based on time windows and alarm attributes at each alarm level. Specifically, as Figure 5 As shown in the figure, the aggregation can be based on the system dimension and the alarm monitoring tool dimension: the alarms of the same system components generated by the same monitoring tool can be aggregated, and multiple alarms can be aggregated into one alarm. Figure 5 As shown, the three alarm messages are all from component A under cluster A of system A, and are all from the "zabbix" alarm monitoring tool. Therefore, after aggregation, the aggregated alarm title of system A component A cluster A, the aggregated alarm content of system A component A cluster A, as well as information such as alarm monitoring tools and IP addresses are obtained. In another embodiment of the present invention, deduplication aggregation can also be performed based on the time window combined with the alarm attribute: in the actual practice of the embodiment of the present invention, an open source memory computing framework esper stream processing framework based on Java is applied to implement advanced analysis operations such as filtering and aggregation on real-time data streams. Aggregation is performed based on the time window and the title attribute and IP attribute of the alarm, including the following steps: constructing an aggregation query statement, creating a stream processing context statement, starting a stream processing monitoring statement, creating a context and starting the monitoring related code. Specifically, if a stream processing statement is created, the alarms with the same IP and title within 3 seconds are aggregated, a stream processing context is created, and a stream processing monitoring is started. When the alarm information that meets the conditions comes in, it will be aggregated.

[0073] The preprocessing module 240 is also used to send the compressed multi-dimensional alarm message to the centralized alarm processing platform, so that the centralized alarm processing platform performs centralized processing according to the alarm level. Among them, the embodiment of the present invention can execute different alarm level sending strategies according to the alarm level. For example, high-level multi-dimensional alarm messages are sent to the centralized alarm processing platform first. By setting different levels of message queues and different levels of preprocessing modules, the alarm messages can be preprocessed separately according to the level, and the alarm information is pushed to different message queues according to the level for separate processing after rectification and current limiting, avoiding the impact of the centralized alarm platform directly facing the alarm storm, and effectively ensuring the stability of the centralized alarm platform. Further, by classifying and grading the alarms, and preprocessing the alarms with different strategies according to the alarm level, the operation and maintenance personnel can understand the nature and urgency of the alarm more clearly, so that the key alarms are displayed first and the interference of useless alarms is reduced. In addition, the embodiment of the present invention adopts technical means such as alarm deduplication and compression to effectively reduce invalid alarms, reduce alarm noise, and improve the alarm handling efficiency of the operation and maintenance personnel.

[0074] The centralized alarm processing platform processes alarms according to the preset alarm processing strategy, such as according to the alarm level priority. For alarm messages of different alarm levels and priorities, the priority can be further set according to the alarm type, and then priority processing is performed. The setting of the intermediate platform reduces the processing level of alarm messages received by the centralized alarm processing platform, and enriches the alarm messages in advance on the intermediate platform, making the alarm message description more accurate, effectively reducing the risk of information overload and improving the alarm processing capability.

[0075] like Figure 2 As shown, it is a structural diagram of the alarm processing device of another embodiment of the present invention. Among them, n alarm monitoring tools monitor the original alarm messages of different monitoring items of n application systems. The original alarm message is received through the logstash pipeline, and after the original alarm message is processed by current limiting and rectification, it is sent to the message queues of different alarm levels of the kafka message queue according to different levels. The kafka message queue consumes the target alarm messages of different levels based on the alarm level through the preprocessing module, and performs aggregation and compression, and finally sends them to the alarm receiving platform for centralized processing.

[0076] The embodiment of the present invention collects the original alarm messages of each application system through each preset alarm monitoring tool, receives the original alarm messages through the data pipeline cluster, and performs current limiting and rectification processing on the original alarm messages to obtain multiple processed target alarm messages, and sends the processed multiple target alarm messages to the corresponding target message queue according to the corresponding alarm level. Among them, the target message queue is at least one of multiple different message queues set according to the alarm level, and the different message queues include high-level message queues, medium-level message queues and low-level message queues. The alarm level is determined according to the indicator data of the monitoring indicator corresponding to the target alarm message. The target message queue transmits the target alarm message to the preset preprocessing module. The preprocessing module obtains the relevant information from the configuration management database, enriches the target alarm message, and obtains the target alarm message after enrichment. The preprocessing module performs multi-dimensional alarm aggregation compression on the target alarm message after enrichment, obtains the compressed multi-dimensional alarm message, and sends the compressed multi-dimensional alarm message to the centralized alarm processing platform, so that the centralized alarm processing platform performs centralized processing according to the alarm level, thereby rectifying the alarm data through the data pipeline cluster, and consuming the alarm data separately in the queue, so that the alarm message can be decoupled and the stability of the alarm processing can be improved; by setting a hierarchical alarm and using different strategies to process alarms of different levels, it is achieved that high-level alarms are given priority, ensuring that high-level system alarms can be responded to first. Furthermore, the present application significantly reduces the number of alarms and reduces the risk of information overload by setting a variety of compression aggregation strategies.

[0077] Figure 6 The flowchart of the alarm processing method provided by the embodiment of the present invention is shown. The method is executed by the aforementioned alarm processing device, which may be a device set in a computer device. The specific implementation process of the alarm processing method in the embodiment of the present invention is consistent with the specific working process of the alarm processing device in the above embodiment. Figure 6 As shown, the method comprises the following steps:

[0078] Step 110: Collect the original alarm messages of each application system through each preset alarm monitoring tool. Among them, multiple preset alarm monitoring tools 210 are used to collect the original alarm messages of each application system. Specifically, for multiple business systems that need to be monitored, there are multiple different monitoring items inside each business system. Therefore, these monitoring items are monitored by different alarm monitoring tools 210 respectively. Since different business systems need to monitor the underlying resources, machine operation status, log monitoring, business monitoring, etc., different alarm monitoring tools 210 are configured to monitor each business system respectively, so as to collect the original alarm messages of each application system. Among them, multiple preset alarm monitoring tools 210 include zabbix, spark, prometheus, dial-up testing platform, etc. Among them, Zabbix is ​​an enterprise-level open source distributed monitoring solution, which can monitor various network parameters, server health status, application performance and many other indicators. Spark is a fast and general large-scale data processing engine that provides efficient distributed data set operations and supports multiple programming languages, such as Python, Java, Scala, etc. For example, when processing massive amounts of log data, Spark can quickly read, clean, and analyze the data. Prometheus is an open source system monitoring and alerting toolkit that features efficient data collection and a flexible query language for monitoring in cloud-native environments. Dial testing is a method for detecting the availability and performance of network services or applications by simulating user behavior. The original alarm message includes the original alarm title field, the original alarm IP field, the original alarm level field, the application system information field, and so on.

[0079] Step 120: Receive the original alarm message through the data pipeline cluster, and perform flow limiting and rectification processing on the original alarm message to obtain a plurality of processed target alarm messages.

[0080] Among them, before receiving the original alarm message through the data pipeline cluster and performing flow limiting and rectification processing on the original alarm message to obtain multiple processed target alarm messages, first determine the data type of each alarm monitoring tool; according to the data type, select multiple data access plug-ins to establish data transmission protocols with each of the target alarm monitoring tools; the multiple data access plug-ins include at least one or more of HTTP plug-ins, SNMP plug-ins and JDBC plug-ins. After the data transmission protocol is established, receive the original alarm message from each of the target alarm monitoring tools according to the data transmission protocol.

[0081] Wherein, when the data pipeline cluster is limiting the flow, the data pipeline cluster sets a threshold value for the number of original alarm messages received within a preset time; receives the original alarm messages within the threshold value within the preset time; and discards the original alarm message or prints a log when the threshold value is exceeded within the preset time. Wherein, when the data pipeline cluster is rectifying, it extracts target field data from the field information of the original alarm message; determines the attribute information of each original alarm message according to the target field data, the attribute information including the alarm title, alarm content, alarm IP address, alarm occurrence time, alarm level, alarm type and monitoring tool information; adjusts the attribute information according to the attribute format to obtain the target attribute information of the target alarm message.

[0082] Step 130: Send the processed multiple target alarm messages to the corresponding target message queues according to the corresponding alarm levels through the data pipeline cluster.

[0083] Among them, the target message queue is at least one of a plurality of different message queues set according to alarm levels; the different message queues include high-level message queues, medium-level message queues and low-level message queues; the alarm level is determined according to the indicator data of the monitoring indicator corresponding to the target alarm message.

[0084] Step 140: The target message queue transmits the target alarm message to a preset preprocessing module. In the embodiment of the present invention, a kafka queue can be used as a message queue. Specifically, in order to eliminate peak decoupling, the embodiment of the present invention puts the target alarm messages after rectification into different message queues respectively, and consumes them separately in different message queues.

[0085] Step 150: The preprocessing module obtains related information from the configuration management database, and enriches the target alarm message to obtain an information-enriched target alarm message.

[0086] Among them, the target level alarm consumer obtains the target association information corresponding to the target alarm information from the configuration management database of the corresponding application system according to the IP information of the target alarm message; the target level alarm consumer is one of the high-level alarm consumer, the medium-level alarm consumer and the low-level alarm consumer; the target association information includes one or more of the computer room information, the application system classification, the application system, component information, cluster information and relevant person in charge information; the target alarm message is enriched according to the target association information to obtain the target alarm message with enriched information.

[0087] Step 160: The preprocessing module performs multi-dimensional alarm aggregation compression on the information-enriched target alarm message to obtain a compressed multi-dimensional alarm message.

[0088] Among them, the embodiment of the present invention, for the information-enriched target alarm messages of different alarm levels, respectively aggregates multiple information-enriched target alarm messages of the same application system component monitored by the same monitoring alarm tool, to obtain aggregated alarm messages based on system dimensions and alarm tool dimensions at each alarm level; for the information-enriched target alarm messages of different alarm levels, respectively aggregates multiple information-enriched target alarm messages with the same IP address and alarm title within the same time period, to obtain aggregated alarm messages based on time windows and alarm attributes at each alarm level; the aggregated alarm messages based on system dimensions and alarm tool dimensions at each alarm level and the aggregated alarm messages based on time windows and alarm attributes at each alarm level are used as compressed multi-dimensional alarm messages.

[0089] Step 170: Send the compressed multi-dimensional alarm message to the centralized alarm processing platform, so that the centralized alarm processing platform performs centralized processing according to the alarm level.

[0090] Among them, the centralized alarm processing platform processes according to the preset alarm processing strategy, such as processing according to the alarm level priority. For alarm messages of different alarm levels and priorities, further priority settings can be made according to the alarm type, etc., and then priority processing is performed. The setting of the intermediate platform reduces the processing level of alarm messages received by the centralized alarm processing platform, and enriches the information of the alarm messages in advance on the intermediate platform, making the alarm message description more accurate, effectively reducing the risk of information overload and improving the alarm processing capability.

[0091] The embodiment of the present invention collects the original alarm messages of each application system through each preset alarm monitoring tool, receives the original alarm messages through the data pipeline cluster, and performs current limiting and rectification processing on the original alarm messages to obtain multiple processed target alarm messages, and sends the processed multiple target alarm messages to the corresponding target message queue according to the corresponding alarm level. Among them, the target message queue is at least one of multiple different message queues set according to the alarm level, and the different message queues include high-level message queues, medium-level message queues and low-level message queues. The alarm level is determined according to the indicator data of the monitoring indicator corresponding to the target alarm message. The target message queue transmits the target alarm message to the preset preprocessing module. The preprocessing module obtains the relevant information from the configuration management database, enriches the target alarm message, and obtains the target alarm message after enrichment. The preprocessing module performs multi-dimensional alarm aggregation compression on the target alarm message after enrichment, obtains the compressed multi-dimensional alarm message, and sends the compressed multi-dimensional alarm message to the centralized alarm processing platform, so that the centralized alarm processing platform performs centralized processing according to the alarm level, thereby rectifying the alarm data through the data pipeline cluster, and consuming the alarm data separately in the queue, so that the alarm message can be decoupled and the stability of the alarm processing can be improved; by setting a hierarchical alarm and using different strategies to process alarms of different levels, it is achieved that high-level alarms are given priority, ensuring that high-level system alarms can be responded to first. Furthermore, the present application significantly reduces the number of alarms and reduces the risk of information overload by setting a variety of compression aggregation strategies.

[0092] Figure 7 The schematic diagram of the structure of the computer device provided by the embodiment of the present invention is shown. The specific embodiment of the present invention does not limit the specific implementation of the computer device.

[0093] like Figure 7 As shown, the computer device may include: a processor (processor) 402 , a communication interface (Communications Interface) 404 , a memory (memory) 406 , and a communication bus 408 .

[0094] The processor 402, the communication interface 404, and the memory 406 communicate with each other via the communication bus 408. The communication interface 404 is used to communicate with other devices such as a client or other server network elements. The processor 402 is used to execute the program 410, which can specifically execute the relevant steps in the above-mentioned embodiment of the alarm processing method.

[0095] Specifically, the program 410 may include program code including computer executable instructions.

[0096] The processor 402 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in the computer device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0097] The memory 406 is used to store the program 410. The memory 406 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0098] The program 410 may be specifically called by the processor 402 to enable the computer device to perform the following operations:

[0099] Collect original alarm messages from each application system through various preset alarm monitoring tools;

[0100] The original alarm message is received through a data pipeline cluster, and current limiting and rectification processing is performed on the original alarm message to obtain a plurality of processed target alarm messages;

[0101] Sending the processed multiple target alarm messages to corresponding target message queues according to corresponding alarm levels through the data pipeline cluster; the target message queue is at least one of multiple different message queues set according to the alarm level; the different message queues include high-level message queues, medium-level message queues and low-level message queues; the alarm level is determined according to the indicator data of the monitoring indicator corresponding to the target alarm message;

[0102] The target message queue transmits the target alarm message to a preset preprocessing module;

[0103] The preprocessing module obtains relevant information from the configuration management database, and enriches the target warning message to obtain an information-enriched target warning message;

[0104] The preprocessing module performs multi-dimensional alarm aggregation compression on the target alarm message after information enrichment to obtain a compressed multi-dimensional alarm message;

[0105] The compressed multi-dimensional alarm message is sent to the centralized alarm processing platform so that the centralized alarm processing platform performs centralized processing according to the alarm level.

[0106] In an optional manner, before receiving the original alarm message through the data pipeline cluster and performing flow limiting and rectification processing on the original alarm message to obtain a plurality of processed target alarm messages, the method includes:

[0107] Determine the data type of each alarm monitoring tool;

[0108] According to the data type, multiple data access plug-ins are selected to establish data transmission protocols with each of the target alarm monitoring tools; the multiple data access plug-ins include at least one or more of HTTP plug-ins, SNMP plug-ins and JDBC plug-ins;

[0109] The receiving the original alarm message through the data pipeline cluster, and performing current limiting and rectification processing on the original alarm message to obtain a plurality of processed target alarm messages, including:

[0110] The original alarm message is received from each of the target alarm monitoring tools according to the data transmission protocol.

[0111] In an optional manner, the receiving the original alarm message through the data pipeline cluster, and performing flow limiting and rectification processing on the original alarm message to obtain a plurality of processed target alarm messages further includes:

[0112] The data pipeline cluster sets a threshold value of the number of the original alarm messages received within a preset time;

[0113] Receiving the original warning message within the quantity threshold within the preset time;

[0114] When the quantity threshold is exceeded within the preset time, the original alarm message is discarded or the log is printed without entering the subsequent processing flow.

[0115] In an optional manner, the receiving the original alarm message through the data pipeline cluster, and performing flow limiting and rectification processing on the original alarm message to obtain a plurality of processed target alarm messages further includes:

[0116] Extracting target field data from the field information of the original alarm message;

[0117] Determine attribute information of each of the original alarm messages according to the target field data, the attribute information including alarm title, alarm content, alarm IP address, alarm occurrence time, alarm level, alarm type and monitoring tool information;

[0118] According to the attribute information, the target attribute information of the target alarm message is obtained by adjusting the attribute format.

[0119] In an optional manner, the preprocessing module includes a high-level alarm consumer, a medium-level alarm consumer, and a low-level alarm consumer; the preprocessing module obtains related information from a configuration management database, enriches the target alarm message, and obtains the target alarm message after enrichment, including:

[0120] The target level alarm consumer obtains the target association information corresponding to the target alarm information from the configuration management database of the corresponding application system according to the IP information of the target alarm message; the target level alarm consumer is one of the high level alarm consumer, the medium level alarm consumer and the low level alarm consumer; the target association information includes one or more of the computer room information, the application system classification, the application system, the component information, the cluster information and the relevant person in charge information;

[0121] The target warning message is enriched according to the target association information to obtain an information-enriched target warning message.

[0122] In an optional manner, the preprocessing module performs multi-dimensional alarm aggregation compression on the information-enriched target alarm message to obtain a compressed multi-dimensional alarm message, including:

[0123] For the target alarm messages with enriched information of different alarm levels, multiple target alarm messages with enriched information of the same application system component monitored by the same monitoring alarm tool are respectively aggregated to obtain aggregated alarm messages based on system dimensions and alarm tool dimensions at each alarm level;

[0124] For the target alarm messages with enriched information of different alarm levels, multiple target alarm messages with enriched information of the same IP address and alarm title within the same time period are aggregated to obtain aggregated alarm messages based on time windows and alarm attributes at each alarm level;

[0125] The aggregated alarm messages based on the system dimension and the alarm tool dimension at each alarm level and the aggregated alarm messages based on the time window and the alarm attribute at each alarm level are used as compressed multi-dimensional alarm messages.

[0126] In an optional manner, for the target alarm messages with enriched information of different alarm levels, multiple target alarm messages with enriched information of the same component under the same application system monitored by the same monitoring alarm tool are respectively aggregated to obtain aggregated alarm messages based on system dimensions and alarm tool dimensions at each alarm level, including: for the target alarm messages with enriched information of different alarm levels, according to different alarm compression strategies, multiple target alarm messages with enriched information of the same component of the same application system monitored by the same monitoring alarm tool are respectively aggregated to obtain aggregated alarm messages based on system dimensions and alarm tool dimensions at each alarm level;

[0127] For the information-enriched target alarm messages of different alarm levels, the multiple information-enriched target alarm messages with the same IP address and alarm title within the same time period are aggregated respectively to obtain the aggregated alarm message based on the time window and alarm attribute at each alarm level, including: for the information-enriched target alarm messages of different alarm levels, according to different alarm compression strategies, the multiple information-enriched target alarm messages with the same IP address and alarm title within the same time period are aggregated respectively to obtain the aggregated alarm message based on the time window and alarm attribute at each alarm level.

[0128] The embodiment of the present invention collects the original alarm messages of each application system through each preset alarm monitoring tool, receives the original alarm messages through the data pipeline cluster, and performs current limiting and rectification processing on the original alarm messages to obtain multiple processed target alarm messages, and sends the processed multiple target alarm messages to the corresponding target message queue according to the corresponding alarm level. Among them, the target message queue is at least one of multiple different message queues set according to the alarm level, and the different message queues include high-level message queues, medium-level message queues and low-level message queues. The alarm level is determined according to the indicator data of the monitoring indicator corresponding to the target alarm message. The target message queue transmits the target alarm message to the preset preprocessing module. The preprocessing module obtains the relevant information from the configuration management database, enriches the target alarm message, and obtains the target alarm message after enrichment. The preprocessing module performs multi-dimensional alarm aggregation compression on the target alarm message after enrichment, obtains the compressed multi-dimensional alarm message, and sends the compressed multi-dimensional alarm message to the centralized alarm processing platform, so that the centralized alarm processing platform performs centralized processing according to the alarm level, thereby rectifying the alarm data through the data pipeline cluster, and consuming the alarm data separately in the queue, so that the alarm message can be decoupled and the stability of the alarm processing can be improved; by setting a hierarchical alarm and using different strategies to process alarms of different levels, it is achieved that high-level alarms are given priority, ensuring that high-level system alarms can be responded to first. Furthermore, the present application significantly reduces the number of alarms and reduces the risk of information overload by setting a variety of compression aggregation strategies.

[0129] An embodiment of the present invention provides a computer-readable storage medium, wherein the storage medium stores at least one executable instruction. When the executable instruction is executed on a computer device, the computer device executes the alarm processing method in any of the above method embodiments.

[0130] The executable instructions can be specifically used to cause the computer device to perform the following operations:

[0131] Collect original alarm messages from each application system through various preset alarm monitoring tools;

[0132] The original alarm message is received through a data pipeline cluster, and current limiting and rectification processing is performed on the original alarm message to obtain a plurality of processed target alarm messages;

[0133] Sending the processed multiple target alarm messages to corresponding target message queues according to corresponding alarm levels through the data pipeline cluster; the target message queue is at least one of multiple different message queues set according to the alarm level; the different message queues include high-level message queues, medium-level message queues and low-level message queues; the alarm level is determined according to the indicator data of the monitoring indicator corresponding to the target alarm message;

[0134] The target message queue transmits the target alarm message to a preset preprocessing module;

[0135] The preprocessing module obtains the associated information from the configuration management database, and enriches the target warning message to obtain the target warning message after enrichment;

[0136] The preprocessing module performs multi-dimensional alarm aggregation compression on the target alarm message after information enrichment to obtain a compressed multi-dimensional alarm message;

[0137] The compressed multi-dimensional alarm message is sent to a centralized alarm processing platform so that the centralized alarm processing platform performs centralized processing according to the alarm level.

[0138] In an optional manner, before receiving the original alarm message through the data pipeline cluster and performing flow limiting and rectification processing on the original alarm message to obtain a plurality of processed target alarm messages, the method includes:

[0139] Determine the data type of each alarm monitoring tool;

[0140] According to the data type, multiple data access plug-ins are selected to establish data transmission protocols with each of the target alarm monitoring tools; the multiple data access plug-ins include at least one or more of HTTP plug-ins, SNMP plug-ins and JDBC plug-ins;

[0141] The receiving the original alarm message through the data pipeline cluster, and performing current limiting and rectification processing on the original alarm message to obtain a plurality of processed target alarm messages, including:

[0142] The original alarm message is received from each of the target alarm monitoring tools according to the data transmission protocol.

[0143] In an optional manner, the receiving the original alarm message through the data pipeline cluster, and performing flow limiting and rectification processing on the original alarm message to obtain a plurality of processed target alarm messages further includes:

[0144] The data pipeline cluster sets a threshold value of the number of the original alarm messages received within a preset time;

[0145] Receiving the original alarm message with the quantity threshold within the preset time;

[0146] When the quantity threshold is exceeded within the preset time, the original alarm message is discarded or the log is printed without entering the subsequent processing flow.

[0147] In an optional manner, the receiving the original alarm message through the data pipeline cluster, and performing flow limiting and rectification processing on the original alarm message to obtain a plurality of processed target alarm messages further includes:

[0148] Extracting target field data from the field information of the original alarm message;

[0149] Determine attribute information of each of the original alarm messages according to the target field data, the attribute information including alarm title, alarm content, alarm IP address, alarm occurrence time, alarm level, alarm type and monitoring tool information;

[0150] According to the attribute information, the target attribute information of the target alarm message is obtained by adjusting the attribute format.

[0151] In an optional manner, the preprocessing module includes a high-level alarm consumer, a medium-level alarm consumer, and a low-level alarm consumer; the preprocessing module obtains related information from a configuration management database, enriches the target alarm message, and obtains the target alarm message after enrichment, including:

[0152] The target level alarm consumer obtains the target association information corresponding to the target alarm information from the configuration management database of the corresponding application system according to the IP information of the target alarm message; the target level alarm consumer is one of the high level alarm consumer, the medium level alarm consumer and the low level alarm consumer; the target association information includes one or more of the computer room information, the application system classification, the application system, the component information, the cluster information and the relevant person in charge information;

[0153] The target warning message is enriched according to the target association information to obtain an information-enriched target warning message.

[0154] In an optional manner, the preprocessing module performs multi-dimensional alarm aggregation compression on the information-enriched target alarm message to obtain a compressed multi-dimensional alarm message, including:

[0155] For the target alarm messages with enriched information of different alarm levels, multiple target alarm messages with enriched information of the same application system component monitored by the same monitoring alarm tool are respectively aggregated to obtain aggregated alarm messages based on system dimensions and alarm tool dimensions at each alarm level;

[0156] For the target alarm messages with enriched information of different alarm levels, multiple target alarm messages with enriched information of the same IP address and alarm title within the same time period are aggregated to obtain aggregated alarm messages based on time windows and alarm attributes at each alarm level;

[0157] The aggregated alarm messages based on the system dimension and the alarm tool dimension at each alarm level and the aggregated alarm messages based on the time window and the alarm attribute at each alarm level are used as compressed multi-dimensional alarm messages.

[0158] In an optional manner, for the target alarm messages with enriched information of different alarm levels, multiple target alarm messages with enriched information of the same component under the same application system monitored by the same monitoring alarm tool are respectively aggregated to obtain aggregated alarm messages based on system dimensions and alarm tool dimensions at each alarm level, including: for the target alarm messages with enriched information of different alarm levels, according to different alarm compression strategies, multiple target alarm messages with enriched information of the same component of the same application system monitored by the same monitoring alarm tool are respectively aggregated to obtain aggregated alarm messages based on system dimensions and alarm tool dimensions at each alarm level;

[0159] For the information-enriched target alarm messages of different alarm levels, the multiple information-enriched target alarm messages with the same IP address and alarm title within the same time period are aggregated respectively to obtain the aggregated alarm message based on the time window and alarm attribute at each alarm level, including: for the information-enriched target alarm messages of different alarm levels, according to different alarm compression strategies, the multiple information-enriched target alarm messages with the same IP address and alarm title within the same time period are aggregated respectively to obtain the aggregated alarm message based on the time window and alarm attribute at each alarm level.

[0160] The embodiment of the present invention collects the original alarm messages of each application system through each preset alarm monitoring tool, receives the original alarm messages through the data pipeline cluster, and performs current limiting and rectification processing on the original alarm messages to obtain multiple processed target alarm messages, and sends the processed multiple target alarm messages to the corresponding target message queue according to the corresponding alarm level. Among them, the target message queue is at least one of multiple different message queues set according to the alarm level, and the different message queues include high-level message queues, medium-level message queues and low-level message queues. The alarm level is determined according to the indicator data of the monitoring indicator corresponding to the target alarm message. The target message queue transmits the target alarm message to the preset preprocessing module. The preprocessing module obtains the relevant information from the configuration management database, enriches the target alarm message, and obtains the target alarm message after enrichment. The preprocessing module performs multi-dimensional alarm aggregation compression on the target alarm message after enrichment, obtains the compressed multi-dimensional alarm message, and sends the compressed multi-dimensional alarm message to the centralized alarm processing platform, so that the centralized alarm processing platform performs centralized processing according to the alarm level, thereby rectifying the alarm data through the data pipeline cluster, and consuming the alarm data separately in the queue, so that the alarm message can be decoupled and the stability of the alarm processing can be improved; by setting a hierarchical alarm and using different strategies to process alarms of different levels, it is achieved that high-level alarms are given priority, ensuring that high-level system alarms can be responded to first. Furthermore, the present application significantly reduces the number of alarms and reduces the risk of information overload by setting a variety of compression aggregation strategies.

[0161] The algorithm or display provided herein is not inherently related to any particular computer, virtual system or other equipment. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious to construct the structure required for this type of system. In addition, the embodiment of the present invention is not directed to any specific programming language yet. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the description made to specific languages ​​above is for disclosing the best mode of the present invention.

[0162] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0163] Similarly, it should be understood that in order to streamline the present invention and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the embodiments of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this method of disclosure should not be interpreted as reflecting an intention that the claimed invention requires more features than those expressly recited in each claim.

[0164] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0165] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be understood as limitations on the order of execution.

Claims

1. An alarm processing method, characterized in that: The method comprises: Collect original alarm messages from each application system through various preset alarm monitoring tools; The original alarm message is received through a data pipeline cluster, and current limiting and rectification processing is performed on the original alarm message to obtain a plurality of processed target alarm messages; Sending the processed multiple target alarm messages to corresponding target message queues according to corresponding alarm levels through the data pipeline cluster; the target message queue is at least one of multiple different message queues set according to the alarm level; the different message queues include high-level message queues, medium-level message queues and low-level message queues; the alarm level is determined according to the indicator data of the monitoring indicator corresponding to the target alarm message; The target message queue transmits the target alarm message to a preset preprocessing module; The preprocessing module obtains the associated information from the configuration management database, and enriches the target warning message to obtain the target warning message after enrichment; The preprocessing module performs multi-dimensional alarm aggregation compression on the target alarm message after information enrichment to obtain a compressed multi-dimensional alarm message; The compressed multi-dimensional alarm message is sent to a centralized alarm processing platform so that the centralized alarm processing platform performs centralized processing according to the alarm level.

2. The method according to claim 1, characterized in that Before receiving the original alarm message through the data pipeline cluster and performing current limiting and rectification processing on the original alarm message to obtain a plurality of processed target alarm messages, the method includes: Determine the data type of each alarm monitoring tool; According to the data type, multiple data access plug-ins are selected to establish data transmission protocols with each of the target alarm monitoring tools; the multiple data access plug-ins include at least one or more of HTTP plug-ins, SNMP plug-ins and JDBC plug-ins; The receiving the original alarm message through the data pipeline cluster, and performing current limiting and rectification processing on the original alarm message to obtain a plurality of processed target alarm messages, including: The original alarm message is received from each of the target alarm monitoring tools according to the data transmission protocol.

3. The method according to claim 2, characterized in that The receiving the original alarm message through the data pipeline cluster, and performing current limiting and rectification processing on the original alarm message to obtain a plurality of processed target alarm messages further includes: The data pipeline cluster sets a threshold value of the number of the original alarm messages received within a preset time; Receiving the original warning message within the quantity threshold within the preset time; When the number of original alarm messages within the preset time exceeds the quantity threshold, the original alarm messages exceeding the threshold are discarded or printed into logs without entering into the subsequent processing flow.

4. The method according to any one of claims 1 to 3, characterized in that: The receiving the original alarm message through the data pipeline cluster, and performing current limiting and rectification processing on the original alarm message to obtain a plurality of processed target alarm messages further includes: Extracting target field data from the field information of the original alarm message; Determine attribute information of each of the original alarm messages according to the target field data, the attribute information including alarm title, alarm content, alarm IP address, alarm occurrence time, alarm level, alarm type and monitoring tool information; According to the attribute information, the target attribute information of the target alarm message is obtained by adjusting the attribute format.

5. The method according to claim 4, characterized in that The pre-processing module includes a high-level alarm consumer, a medium-level alarm consumer and a low-level alarm consumer; the pre-processing module obtains related information from the configuration management database, enriches the target alarm message, and obtains the target alarm message after enrichment, including: The target level alarm consumer obtains the target association information corresponding to the target alarm information from the configuration management database according to the IP information of the target alarm message; the target level alarm consumer is one of the high level alarm consumer, the medium level alarm consumer and the low level alarm consumer; the target association information includes one or more of the computer room information, the application system classification, the application system, the component information, the cluster information and the relevant person in charge information; The target warning message is enriched according to the target association information to obtain an information-enriched target warning message.

6. The method according to claim 4, characterized in that The preprocessing module performs multi-dimensional alarm aggregation compression on the target alarm message after enriching the information to obtain a compressed multi-dimensional alarm message, including: For the target alarm messages with enriched information of different alarm levels, multiple target alarm messages with enriched information of the same component under the same application system monitored by the same monitoring alarm tool are aggregated to obtain aggregated alarm messages based on system dimensions and alarm tool dimensions at each alarm level; For the target alarm messages with enriched information of different alarm levels, multiple target alarm messages with enriched information of the same IP address and alarm title within the same time period are aggregated to obtain aggregated alarm messages based on time windows and alarm attributes at each alarm level; The aggregated alarm messages based on the system dimension and the alarm tool dimension at each alarm level and the aggregated alarm messages based on the time window and the alarm attribute at each alarm level are used as compressed multi-dimensional alarm messages.

7. The method according to claim 6, characterized in that The target alarm messages after information enrichment of different alarm levels are respectively aggregating multiple target alarm messages after information enrichment of the same application system component monitored by the same monitoring alarm tool to obtain aggregated alarm messages based on system dimensions and alarm tool dimensions at each alarm level, including: for the target alarm messages after information enrichment of different alarm levels, according to different alarm compression strategies, multiple target alarm messages after information enrichment of the same application system component monitored by the same monitoring alarm tool are respectively aggregating to obtain aggregated alarm messages based on system dimensions and alarm tool dimensions at each alarm level; For the information-enriched target alarm messages of different alarm levels, the multiple information-enriched target alarm messages with the same IP address and alarm title within the same time period are aggregated respectively to obtain the aggregated alarm message based on the time window and alarm attribute at each alarm level, including: for the information-enriched target alarm messages of different alarm levels, according to different alarm compression strategies, the multiple information-enriched target alarm messages with the same IP address and alarm title within the same time period are aggregated respectively to obtain the aggregated alarm message based on the time window and alarm attribute at each alarm level.

8. An alarm processing device, characterized in that: The device comprises: Multiple preset alarm monitoring tools are used to collect original alarm messages from various application systems; A data pipeline cluster is used to receive the original alarm message, and perform current limiting and rectification processing on the original alarm message to obtain a plurality of processed target alarm messages; The data pipeline cluster is further used to send the processed multiple target alarm messages to corresponding target message queues according to corresponding alarm levels; the target message queue is at least one of multiple different message queues set according to the alarm level; the different message queues include a high-level message queue, a medium-level message queue and a low-level message queue; The target message queue is used to transmit the target alarm message to a preset pre-processing module; The preprocessing module is used to obtain relevant information from the configuration management database, enrich the target warning message, and obtain the target warning message after enrichment; The preprocessing module is further used to perform multi-dimensional alarm aggregation compression on the target alarm message after information enrichment to obtain a compressed multi-dimensional alarm message; The pre-processing module is also used to send the compressed multi-dimensional alarm message to the centralized alarm processing platform, so that the centralized alarm processing platform performs centralized processing according to the alarm level.

9. A computer device, characterized in that: include: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform the operation of the alarm processing method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The storage medium stores at least one executable instruction. When the executable instruction is executed on a computer device, the computer device executes the operation of the alarm processing method according to any one of claims 1 to 7.

Images