Alarm data processing method and device, equipment and storage medium
By configuring preset alarm data extraction rules in the configuration file and parsing to generate broadcast streams, the processing complexity problem caused by differences in alarm data formats from different security vendors is solved, and unified and efficient processing of alarm data is achieved.
Patent Information
- Application Number
- CN202411245187.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-05
- Publication Date
- 2025-01-14
AI Technical Summary
In the field of network security, there are significant differences in the alarm data formats of different security manufacturers, which leads to the need to develop customized analysis and processing programs for each format, which increases maintenance complexity and development workload, making it difficult to achieve efficient unified processing.
By configuring preset alarm data extraction rules in the configuration file, parsing these rules to generate alarm data extraction rule broadcast streams, and connecting the original alarm data stream with the broadcast stream, filtering out the target alarm data, and realizing the processing of different alarm data formats unified processing.
It reduces the complexity and development cost of the alarm data processing process, improves the flexibility and efficiency of the processing process, ensures the real-time and accuracy of the alarm data, and can efficiently respond to the standardized access requirements of unknown data.
Smart Images

Figure CN119316186A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to an alarm data processing method, device, equipment and storage medium. Background Art
[0002] In the field of network security, when the alarm data provided by many security vendors are integrated into the security operation center for unified processing, there are significant differences in the formats of log records and alarm information among the products of different security vendors. This diversity requires parsing and conversion of each format when accessing data to meet the requirements of the backend system (such as Elasticsearch).
[0003] Different manufacturers have different alarm strategies and trigger conditions, resulting in great differences in the content and structure of alarm information. Each manufacturer's product needs to independently develop and maintain its alarm data access solution, which makes simple unified processing difficult.
[0004] In traditional solutions, customized parsing and processing programs are often developed for each different alarm data format. This method requires a large amount of code to process different log formats, which increases the complexity of maintenance. Each data connection requires personalized development based on the data format, which not only increases the development workload, but also prolongs the connection cycle, making it difficult to efficiently respond to the standardized access requirements of unknown data. Summary of the invention
[0005] The embodiments of the present application provide a method, an apparatus, a device and a storage medium, which can solve various technical problems.
[0006] In a first aspect, an embodiment of the present application provides a method for processing alarm data, including:
[0007] The original alarm data stream and the preset alarm data extraction rules are obtained. The preset alarm data extraction rules are configured in the configuration file.
[0008] The preset alarm data extraction rules are parsed to obtain the alarm data extraction rule broadcast stream.
[0009] The original alarm data stream and the alarm data extraction rule broadcast stream are connected and processed to obtain an alarm configuration connection stream.
[0010] Target alarm data is filtered out from the alarm configuration connection flow, where the target alarm data is alarm data that meets preset alarm data extraction rules.
[0011] In one implementation, the preset alarm data extraction rule includes a key value extraction rule and / or a conditional extraction rule, and the method further includes:
[0012] Detect identification information of a preset alarm data extraction rule, where the identification information includes key value identification information and / or condition identification information;
[0013] When the key value identification information is detected, the preset alarm data extraction rule is determined as the key value extraction rule; or,
[0014] When the condition identification information is detected, the preset alarm data extraction rule is determined as the condition extraction rule.
[0015] In one implementation, when the alarm data extraction rule broadcast stream is obtained by connecting and processing the key value extraction rule, the target alarm data is filtered out from the alarm configuration connection stream, including:
[0016] Get the target key value configured in the configuration file;
[0017] According to the target key value, a search is performed in the alarm configuration connection flow, and the alarm data whose key value is equal to the target key value is determined as the target alarm data.
[0018] In one implementation, the method further includes:
[0019] Extracting a field equal to a target key value from target alarm data to obtain a target alarm field;
[0020] A target alarm result is generated according to the target alarm field, and the target alarm result is used to instruct the user to perform an alarm operation.
[0021] In one implementation, when the alarm data extraction rule broadcast stream is obtained by connecting and processing the conditional extraction rule, the target alarm data is screened out from the alarm configuration connection stream, including:
[0022] Get the target condition value configured in the configuration file. The target condition value is obtained by at least one of the following matching methods: regular expression matching method, inclusion matching method, and JSON matching method.
[0023] According to the target condition value, a search is performed in the alarm configuration connection flow, and the alarm data satisfying the target condition value is determined as the target alarm data.
[0024] In one implementation, the method further includes:
[0025] Extract the fields that meet the regular expression matching method from the target alarm data to obtain the target alarm field; or,
[0026] Extracting fields that satisfy the inclusion matching method from the target alarm data to obtain the target alarm field; or,
[0027] Extract the fields that meet the JSON matching method from the target alarm data to obtain the target alarm fields;
[0028] A target alarm result is generated according to the target alarm field, and the target alarm result is used to instruct the user to perform an alarm operation.
[0029] In one implementation, the method further includes:
[0030] Standardizing the target alarm data using a standard alarm format to obtain standard alarm data, where the standard alarm format includes at least one of the following: alarm time information, alarm severity information, and alarm triggering device type information;
[0031] Generate standard alarm results based on standard alarm data.
[0032] In a second aspect, an embodiment of the present application provides an alarm data processing device, which has the function of implementing the method in the first aspect or any possible implementation thereof. Specifically, the device includes a unit that implements the method in the first aspect or any possible implementation thereof.
[0033] In one embodiment, the device comprises:
[0034] An acquisition unit, used for acquiring an original alarm data stream and a preset alarm data extraction rule, wherein the preset alarm data extraction rule is obtained by configuring in a configuration file;
[0035] A processing unit, used for parsing the preset alarm data extraction rule to obtain the alarm data extraction rule broadcast stream;
[0036] The processing unit is also used to connect the original alarm data flow and the alarm data extraction rule broadcast flow to obtain an alarm configuration connection flow;
[0037] The processing unit is further used to filter out target alarm data from the alarm configuration connection flow, where the target alarm data is alarm data that meets preset alarm data extraction rules.
[0038] In a third aspect, an embodiment of the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the computer device implements a method according to any one of the implementation modes of the first aspect.
[0039] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a computer device, the computer device implements the method of any one of the implementation methods of the first aspect above.
[0040] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on a computer device, the computer device executes a method of any one of the implementation modes of the first aspect above.
[0041] Compared with the prior art, the embodiments of the present application have the following beneficial effects: by using a configuration file to encapsulate the preset alarm data extraction rules, that is, using the configured configuration file to process the acquired original alarm data stream, compared with the traditional method of designing corresponding codes for each type of alarm data, when the alarm data changes, the method of the present application only needs to modify the corresponding configuration file, and does not need to change the overall framework code, thereby reducing the complexity of the alarm data processing process and improving the flexibility and efficiency of the alarm data processing process; parsing and converting the extraction rules into a format that can be disseminated, realizing real-time sharing with multiple processing modules, and improving the real-time performance of the alarm data processing process; connecting the original alarm data stream and the alarm data extraction rule broadcast stream, the system can match and integrate data in real time, that is, the rules in the broadcast stream can be dynamically updated, and when new rules are released, the connection processing will automatically apply the latest rules to ensure the real-time performance and accuracy of the alarm data; by screening the target alarm data, attention can be focused on truly important security events, avoiding omissions or delayed responses caused by information overload, and improving the processing efficiency of the alarm data processing process. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 It is an application scenario diagram of alarm data processing provided in an embodiment of the present application.
[0043] Figure 2 It is a flowchart of an alarm data processing method provided in an embodiment of the present application.
[0044] Figure 3 It is a flow chart of a method for extracting target alarm fields provided in an embodiment of the present application.
[0045] Figure 4 It is a flowchart of an alarm data processing method provided in an embodiment of the present application.
[0046] Figure 5 It is a structural diagram of an alarm data processing device provided in an embodiment of the present application.
[0047] Figure 6 It is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0048] In the field of network security, the security operation center needs to integrate network security alert data in different formats from different security vendors to achieve unified monitoring and response. The data integration process usually involves multiple steps, including data access, parsing, conversion, storage and analysis.
[0049] Combine the following Figure 1 Let's explain in detail an application scenario of alarm data processing.
[0050] Figure 1 It is an application scenario diagram of alarm data processing provided in an embodiment of the present application.
[0051] like Figure 1 As shown, when the alarm data collected by various security vendors is obtained, for example, it can be alarm data from firewalls, intrusion detection systems (IDS), and antivirus software, and then the security operation center will process it uniformly, identify the log formats of different security products (such as JSON, XML, CSV, Plain Text, etc.), and perform parsing processing to facilitate the back-end Elasticsearch index to perform alarm analysis, identify potential security incidents, and generate alarm results.
[0052] It is understandable that products from different security vendors differ significantly in terms of log formats, field names, data structures, etc. When the backend processes these diverse alarm data, format conversion is required. Traditional solutions often design a set of format conversion methods specifically for each data format. Once a new format appears, the old format conversion method is often difficult to apply and can only be redeveloped, making it difficult to process large amounts of alarm data in real time and efficiently.
[0053] In response to the above problems, the present application proposes an alarm data processing method, which encapsulates the processing logic of the alarm data by using a configuration file, and simultaneously loads and parses the configuration file during the process of processing the alarm data, thereby realizing real-time processing of various alarm data without the need for additional targeted development of new program code.
[0054] In order to further illustrate the technical solution of the present application, specific embodiments are provided below.
[0055] Figure 2 It is a flowchart of an alarm data processing method provided in an embodiment of the present application.
[0056] like Figure 2 As shown, the above method includes the following steps S201 to S204.
[0057] S201. Obtaining an original alarm data stream and preset alarm data extraction rules.
[0058] The raw alarm data stream refers to the unprocessed alarm data collected from various security devices (such as firewalls, intrusion detection systems, anti-virus software, etc.).
[0059] As an example but not limitation, the original alarm data stream may include the following information: alarm time, alarm type, alarm level, device information and possible security event description, etc. The specific content may be adjusted according to actual conditions.
[0060] For example, the raw alarm data generated by a certain firewall can be represented by code as follows:
[0061] {
[0062] "timestamp":"2023-01-01T12:00:00Z",
[0063] "device":"Firewall1",
[0064] "alert_type":"Blocked Port",
[0065] "level":"Critical",
[0066] "description":"Blocked connection attempt on port 23"
[0067] }.
[0068] The preset alarm data extraction rule defines how to extract specific alarm information from the original alarm data, which may include field selection, data format conversion, filtering conditions, etc. The specific extraction rule can be adjusted according to the corresponding alarm data and is not limited here.
[0069] The preset alarm data extraction rules are obtained by configuring in a configuration file, for example, encapsulating the preset alarm data extraction rules in a yaml configuration file.
[0070] S202: parse the preset alarm data extraction rule to obtain the alarm data extraction rule broadcast stream.
[0071] The alarm data extraction rule broadcast stream refers to the result of parsing the preset alarm data extraction rules to form a stream that can be distributed to multiple processing nodes.
[0072] Broadcast streaming can be understood as a streaming technology that distributes alarm data extraction rules to multiple receiving modules or processing units in real time. For example, the extraction rules can be propagated to all relevant components through event-driven or message queue methods. This method ensures that any module can obtain the latest extraction rules in a timely manner.
[0073] It can be understood that parsing the preset alarm data extraction rules enables the preset alarm data extraction rules to be shared and used by various processing modules, which facilitates the centralized management and distributed processing of the preset alarm data extraction rules. In other words, this method can ensure that during the alarm processing process, all modules use consistent extraction rules to maintain the uniformity and consistency of data processing.
[0074] S203: Connect the original alarm data stream and the alarm data extraction rule broadcast stream to obtain an alarm configuration connection stream.
[0075] The alarm configuration connection flow refers to the result of combining the original alarm data flow with the alarm data extraction rule broadcast flow, which can also be understood as the result of applying the preset alarm data extraction rule to the original alarm data. It can also be understood as the process of merging or interacting multiple data streams or data sources in a preset manner.
[0076] It can be understood that by applying the preset alarm data extraction rules to the original alarm data, the alarm data meeting the conditions can be screened out, which is convenient for subsequent analysis and processing.
[0077] S204: Filter out target alarm data from the alarm configuration connection flow.
[0078] The target alarm data is the alarm data that meets the preset alarm data extraction rules.
[0079] In other words, the target alarm data refers to all alarm data that meet the conditions defined by the preset alarm extraction rules, that is, the alarm data set generated after screening.
[0080] It can be understood that extracting the alarm data that really needs attention from the processed data stream facilitates further analysis and response, reduces interference and redundant information, and is conducive to obtaining more accurate alarm results.
[0081] By using a configuration file to encapsulate the preset alarm data extraction rules, and using the configured configuration file to process the obtained original alarm data stream, compared with the traditional method of designing corresponding codes for each type of alarm data, when the alarm data changes, the method of the present application only needs to modify the corresponding configuration file, and does not need to change the overall framework code, which reduces the complexity of the alarm data processing process and improves the flexibility of the alarm data processing process; the extraction rules are parsed and converted into a format that can be transmitted to achieve real-time sharing with multiple processing modules, thereby improving the real-time performance of the alarm data processing process; the original alarm data stream and the alarm data extraction rule broadcast stream are connected and processed, and the system can match and integrate data in real time, that is, the rules in the broadcast stream can be dynamically updated, and when new rules are released, the connection processing will automatically apply the latest rules to ensure the real-time and accuracy of the alarm data; by screening the target alarm data, attention can be focused on truly important security events, avoiding omissions or delayed responses caused by information overload, and improving the processing efficiency of the alarm data processing process.
[0082] In one implementation, the preset alarm data extraction rule includes a key value extraction rule and / or a conditional extraction rule, and the method further includes: detecting identification information of the preset alarm data extraction rule, the identification information including key value identification information and / or conditional identification information; when the key value identification information is detected, determining the preset alarm data extraction rule as the key value extraction rule; or, when the conditional identification information is detected, determining the preset alarm data extraction rule as the conditional extraction rule.
[0083] By searching for keywords in the rule (such as "keywords" identification information), it is determined that the rule belongs to a key value extraction rule; by searching for fields related to the condition (such as "condition" identification information), it is determined that the rule belongs to a condition extraction rule. The following is an introduction with specific examples.
[0084] Assume that there is an original alarm json object alarmJson (original alarm data) and two preset alarm data extraction rules extractorRule_1 and extractorRule_2, which can be represented by the following codes respectively.
[0085] Among them, the original alarm json object alarmJson can be expressed in code as follows:
[0086] {"ts":1721935130,
[0087] "uid":"",
[0088] "logtype":"utm",
[0089] "devname":“SONGFOR STA”,
[0090] "ds":“20240726",
[0091] "hh":"03",
[0092] "vendor":"sangfor",
[0093] "appproto":"",
[0094] "transproto":"TCP",
[0095] "logid":21935130974657,
[0096] “src_1p”:"100.0.8.147",
[0097] "dst_ip":"157.245.134.176",
[0098] "iptype":4,
[0099] "src_port":51399,
[0100] "dst_port":443,
[0101] "nSid":0,
[0102] "nLevel":2,
[0103] "nNetAction":1,
[0104] "nAttackType":3,
[0105] "nSubAttackType":e,
[0106] "nPolicyId":835508684,
[0107] "nAttackCount":0,
[0108] "nLen":398,
[0109] "szRecordTime":"03:18:50",
[0110] "szResult":"<?xm1 version-\"1.0\”
[0111] encoding-\"utf-8\~?>\n<root> \n <url><h2 style=";text-align:left;direction:ltr">157.245,134.176:443 / lat-tst?tkn-948efc7c\n<dep ict><h2 style=";text-align:left;direction:ltr"> 发现病毒:
[0112] Win32 / Trojan.Gen.PK< / sub_desc><req_head>HEAD / lat-tst?tkn*940efc7cHTTP / 1.1\r\nHost:157.245.134.176:443\r\nUser-Agent:snr\r\nConnection:close\r\n\r\n< / req_head><req_body>< / req_body>\n<src_mac>04:7:16:d0:e1:80< / src_mac>\n<dst_mac>14:51:7e:e6:96:01< / dst_mac>\n< / roo t>\n"}。
[0113] 其中,预设告警数据提取规则extractorRule_1可以用代码表示为:
[0114] {time:
[0115] keywords:
[0116] -ts
[0117] -time}。
[0118] 其中,预设告警数据提取规则extractorRule_2可以用代码表示为:
[0119] {srcMAC:
[0120] selection_1:
[0121] result|re:<src_mac>(.*?)< / src_mac>
[0122] selection_2:
[0123] szResult|re:<src_mac>(.*?)< / src_mac>
[0124] selection_3:
[0125] keywords|contain:
[0126] -sender_mac
[0127] -src_mac
[0128] selection_4:
[0129] source|json:mac
[0130] condition:selection_1or selection_2or selection_3or selection_4}。
[0131] 获取到这些代码之后,将预设告警数据提取规则由Object解析成LinkedHashMap类型,解析后的代码可以表示如下。
[0132] 其中,解析后的预设告警数据提取规则extractorRule_1可以用代码表示为:
[0133] {"time":{
[0134] "keywords":
[0135] ["ts","time"]}。
[0136] 其中,解析后的预设告警数据提取规则extractorRule_2可以用代码表示为:
[0137] {"srcMAC":{
[0138] "selection_1":{
[0139] "result|re":"<src_mac>(.*?)< / src_mac>"
[0140] },
[0141] "selection_2":{
[0142] "szResult|re":"<src_mac>(.*?)< / src_mac>"},
[0143] "selection_3":{
[0144] "keywords|contain":["sender_mac","src_mac"]},
[0145] "selection_4":{
[0146] "source|json":"mac"},
[0147] "condition":"selection_1or selection_2or selection_3or selection_4"}。
[0148] 结合相关原理可以理解,LinkedHashMap类型对象的第一层级key表示标准告警格式的关键字名称standKey,即根据预设告警数据提取规则从原始告警数据中提取所需数据后要赋值的标准告警格式的字段名。
[0149] 结合上述代码,可以看出extractorRule_1和extractorRule_2的standKey是”time”和”srcMAC”。
[0150] 结合相关原理可以理解,LinkedHashMap类型对象的第一层级value表示具体的内容提取规则matchRule,往往同样是LinkedHashMap类型。
[0151] 可以明显看出,预设告警数据提取规则extractorRule_1的matchRule为:{"keywords”:["ts”,”time”]}。
[0152] 预设告警数据提取规则extractorRule_2的matchRule为:
[0153] {"selection_1":{
[0154] "result|re":"<src_mac>(.*?)< / src_mac>"},
[0155] "selection_2":{
[0156] "szResult|re";"<src_mac>(.*?)< / src_mac>"},
[0157] "selection_3":{
[0158] "keywords|contain":["sender_mac","src_mac"]},
[0159] "selection_4":{
[0160] "source|json";"mac"},
[0161] "condition":"selection_1or selection_2or selection 3or selection_4"}。
[0162] 可以理解,如果matchRule含有”keywords”关键字,则判定该预设告警数据提取规则为关键值提取规则(也可以称为keywords类型规则);如果matchRule含有”condition”关键字,则判定该预设告警数据提取规则为条件提取规则(也可以称为condition类型规则)。
[0163] 可以明显看出,在上面的例子中,预设告警数据提取规则extractorRule_1被判定为关键值提取规则(keywords类型规则);预设告警数据提取规则extractorRule_2被判定为条件提取规则(condition类型规则)。
[0164] 同理,如果在实际应用中遇到其他的预设告警数据提取规则时,可以依据上文中提到的方法进行一一判断,判断的原理是相同,此处不再一一列举。
[0165] 可以理解,无论预设告警数据提取规则是关键值提取规则还是条件提取规则,结合标识信息进行检测,系统会智能地应用这些提取规则到原始告警数据流中,从而生成筛选后的告警数据,供后续分析和决策使用。
[0166] 通过将预设告警数据提取规则分为关键值提取和条件提取,在后续处理告警数据时可以起到一个"分流”的作用,不同的告警数据利用不同的预设告警数据提取规则来进行处理,能够更加高效、灵活地处理原始告警数据。这种分类方法可以帮助分析人员根据需要选择不同的提取方式,使得告警数据的处理过程更加精细化和针对性,提升了响应和管理的效率。
[0167] 在一种实现方式中,当告警数据提取规则广播流是对关键值提取规则进行连接处理得到的时,从告警配置连接流中筛选出目标告警数据,包括:获取在配置文件中配置的目标关键值;根据目标关键值在告警配置连接流中进行遍历查找,将关键值与目标关键值相等的告警数据确定为目标告警数据。
[0168] 目标关键值是指在告警配置中预先定义的一组关键值字段,例如"时间戳”,"设备类型”或者"告警级别”等特定值。具体的目标关键值可以根据实际情况来进行自定义,此处不作限定。
[0169] 在一个例子中,可以通过读取配置文件的方式(如JSON、YAML、XML等格式)从配置文件中解析出目标关键值,获得相关字段并存储为列表或字典。
[0170] 遍历可以理解为是指通过循环结构对数据集进行逐一检查的过程。
[0171] 在一个例子中,对告警配置连接流中的每条告警数据进行循环检查,逐一比较每个告警记录是否包含目标关键值,例如可以使用嵌套循环或者数据映射技术来实现。逐一检查每个告警数据,以找到与目标关键值相匹配的告警数据。这种逐个匹配的方式允许系统精准地从大量数据中提取所需的元素。
[0172] 目标告警数据即成功匹配目标关键值的告警数据。在遍历过程中,若某条告警数据的关键值与目标关键值相等,则将该告警数据标记为目标告警数据。可以使用数组或集合来存储这些匹配的记录,便于后续操作。完成筛选过程后,形成一个新的告警数据子集,后续的分析、报道或警告可以基于这些目标告警数据展开,有效集中对重点告警的处理,提高响应效率。
[0173] 下面结合具体的示例介绍一种利用关键值提取规则来筛选目标告警数据的方法。
[0174] 首先介绍一种可能的关键值提取规则,可以用代码表示如下。
[0175] {alertType:
[0176] keywords:
[0177] category
[0178] AttackType
[0179] attack_type
[0180] nAttackType
[0181] risk_type}。
[0182] 根据上述代码可以理解,关键值包括:category、AttackType、attack_type、nAttackType和risk_type几种。
[0183] 根据这些关键值,遍历JSONObject类型的原始告警数据的键。如果键名为category、AttackType、attack_type、nAttackType或risk_type之一,则视为匹配成功,也就是说将该告警数据确定为目标告警数据。
[0184] 通过将告警配置连接流中的告警数据与目标关键值进行查找比对,可以实现高效的目标告警筛选。这种方式不仅帮助自动化告警管理,提高了响应速度,还减少了潜在的信息过载,为告警决策支持提供了准确基础。
[0185] 在一种实现方式中,从目标告警数据中提取与目标关键值相等的字段,得到目标告警字段;根据目标告警字段生成目标告警结果,目标告警结果用于指示用户执行告警操作。
[0186] 结合上面的例子可以理解,当确定出那条告警数据是目标告警数据之后,为了进一步的分析处理,还可以将对应的关键值提取出来作为标准告警格式中alertType字段的值。
[0187] 在一种实现方式中,当告警数据提取规则广播流是对条件提取规则进行连接处理得到的时,从告警配置连接流中筛选出目标告警数据,包括:获取在配置文件中配置的目标条件值,目标条件值是通过以下至少一项匹配方式得到的:正则表达式匹配方式、包含匹配方式和JSON匹配方式;根据目标条件值在告警配置连接流中进行遍历查找,将满足目标条件值的告警数据确定为目标告警数据。
[0188] 可以理解,相较于基于关键词的内容提取,条件提取规则(condition类型提取规则)对原始告警数据的匹配更为灵活,其特点在于规则明确包含"condition”关键字。
[0189] 这类规则可以通过正则表达式、包含(contain)匹配和JSON匹配三种方式工作,这三种方式在yaml配置文件中通过selection_x层级节点下的xxx|match_way关键字的match_way进行区分。
[0190] 下面结合具体的示例来进行介绍。
[0191] (1)正则表达式匹配方式
[0192] 若原始告警数据中的内容遵循某种模式,则可通过正则表达式确定目标告警数据。
[0193] 以下是一个在yaml配置文件中使用正则表达式提取规则的示例(可以看到result|re中re表示正则表达式):
[0194] {destMAC:
[0195] selection_1:
[0196] result|re:<dst_mac>(.*?)< / dst_mac>
[0197] condition:selection_1}。
[0198] 可以理解,上面的代码意为从JSONObject格式的原始告警数据中提取key为'result'的String类型值,接着通过正则表达式<dst_mac>(.*?)< / dst_mac>与之匹配,如果成功匹配,则认定该条告警数据是目标告警数据。
[0199] (2)JSON匹配方式
[0200] 如果告警标准告警格式的内容在原始告警数据中位于json中,此时只需首先从原始告警数据中获取该json对象,然后从json中获取所需的内容即可。可以用代码表示如下:
[0201] {destIP:
[0202] selection_1:
[0203] destination|json:ip
[0204] condition:selection_1}。
[0205] 可以理解,上面的代码意为从原始告警数据(JSONObject类型)中获取键为"result”的值(一个JSON对象),然后从此对象中提取键为"ip”的值,如果能够提取成功,即意为匹配成功,则认定该条告警数据是目标告警数据。
[0206] (3)包含匹配方式
[0207] 包含(Contain)匹配方式本质上是一种基于关键词的内容提取方法,它与上文中提到的关键值提取规则(keywords类型提取规则)的提取逻辑相同。其归类于条件提取规则(condition类型提取规则)的提取规则是因为标准告警格式的字段值需要适应多样的匹配方式,包括Contain匹配、正则表达式匹配和JSON匹配。下面结合具体的代码来进行介绍。
[0208] {srcMAC:
[0209] selection_1:
[0210] result|re:<src_mac>(.*?)< / src_mac>
[0211] selection_2:
[0212] szResult|re:<src_mac>(.*?)< / src_mac>
[0213] selection_3:
[0214] keywords|contain:
[0215] -sender_mac
[0216] src_mac
[0217] selection_4:
[0218] source|json:mac
[0219] condition:selection_1or selection_2or selection_3or selection_4}。
[0220] 可以理解,上面的代码意为检查原始告警数据中是否包含sender_mac或src_mac的关键值,如存在,则认定匹配成功,即认定该条告警数据是目标告警数据;若不存在,则进行下一规则判断。
[0221] 下面结合上文中预设告警数据提取规则extractorRule_2的matchRule的代码介绍如何确定具体的匹配方式。
[0222] 结合上文,当一个预设告警数据提取规则matchRule被判定为condition类型后,我们需要对该matchRule中的每一个提取条件和告警的匹配方式进行判断,以"selection_1”,"selection_3”和"selection_4”对应的代码为例,具体的判断逻辑如下:
[0223] 对"selection_1”下的map对象{"result|re”:"<src_mac>(.*?)< / src_mac>"}的key"result|re”根据”|”号进行分割,分割后形成列表splits=["result”,”re”],其中splits[1]=”re”即为正则表达式匹配方式。
[0224] 同理可得,"selection_3”和"selection_4”和告警分别为包含(Contain)匹配方式和JSON匹配方式。
[0225] 在一种实现方式中,从目标告警数据中提取满足正则表达式匹配方式的字段,得到目标告警字段;或者,从目标告警数据中提取满足包含匹配方式的字段,得到目标告警字段;或者,从目标告警数据中提取满足JSON匹配方式的字段,得到目标告警字段;根据目标告警字段生成目标告警结果,目标告警结果用于指示用户执行告警操作。
[0226] 结合上文中的例子,可以理解,当满足正则表达式匹配方式时,可以提取对应的字段作为标准告警格式的destMAC字段值,即得到目标告警字段;当满足JSON匹配方式时,可以提取JOSN对象中提取键为"ip”的值,作为标准告警格式中的"destIP”字段值,即得到目标告警字段;当满足包含匹配方式时,可以提取包含sender_mac或src_mac的key对应的值,即得到目标告警字段。
[0227] 根据目标告警字段生成标准告警结果格式,即生成目标告警结果,然后将生成的目标告警结果通知至相关用户或系统组件。例如,可以通过用户界面、电子邮件、短信或其他方式进行通知,具体的告警操作方式此处不作限定。
[0228] 下面结合上文中提到的原始告警json对象alarmJson(原始告警数据)和2个预设告警数据提取规则extractorRule_1及extractorRule_2对应的LinkedHashMap形式下的代码进行说明如何进行目标告警字段提取。
[0229] 结合上文,可知预设告警数据提取规则extractorRule_1为keywords类型规则,将预设告警数据提取规则extractorRule_1转换成LinkedHashMap形式后,将按keywords类型规则从告警对象alarmJson提取目标告警字段。
[0230] 具体流程如下:
[0231] (1)对关键字列表["ts”,"time”]进行遍历,寻找第一个属于告警对象alarmJson的关键字。可以明显看出,第一个属于告警对象alarmJson的关键字是"ts”。
[0232] (2)从alarmJson中提取关键字"ts”对应的值,这里是Long长整型1721935130。
[0233] (3)将提取的值赋给标准告警字段"time”。
[0234] 结合上文,可知预设告警数据提取规则extractorRule_2为condition类型规则,将预设告警数据提取规则extractorRule_2转换成LinkedHashMap形式后,将按condition类型规则从告警对象alarmJson提取目标告警字段。
[0235] 具体流程如下:
[0236] (1)循环遍历预设告警数据提取规则extractorRule_2中的所有匹配条件,寻找第一个能和告警对象alarmJson匹配上的匹配条件。根据告警对象alarmJson和extractorRule_2具体内容可知,第一个匹配条件是selection_2。
[0237] (2)根据上文可知,匹配条件selection_2将按正则表达式匹配方式和告警对象alarmJson进行匹配并内容提取。
[0238] (3)根据selection_2内容,首先从告警对象alarmJson获取key为"szResult”的值,即以下代码:
[0239] "<?xmlversion-\"1.0\"encoding-\"utf-8\”?>\n <root> \n <url> 157.245.134.176:443 / 1at-tst?tk n=940efc7c\n <depict> <sub_desc>发现病毒:
[0240] Win32 / Trojan.Gen.PK< / sub_desc><req_head>HEAD / 1at-tst?tkn=940efc7cHTTP / 1.1\r\nHos t:157.245.134.176:443\r\nUser-Agent:
[0241] snr\r\nConnection:close\r\n\r\n< / req_head><req_body>< / req_body>
[0242] < / depict>\n<src_mac>04:7c:16:d0:e1:80< / src_mac>\n<dst_mac>14:51:7e:e6:96:01< / dst_ma c>\n< / roo t>\n"
[0243] (4)利用selection_2中的正则表达式"<src_mac>(.*?)< / src_mac>”从"szResult”的值提取对应的内容,为"04:7c:16:d0:e1:80”。
[0244] (5)将(4)中提取出的值"04:7c:16:d0:e1:80”赋给标准告警字段"srcMAC”。
[0245] 可以理解,当存在其他告警数据时,按照上文中的相关步骤依次进行处理即可,此处不再一一列举。
[0246] 下面结合 图3来对上文中提到的提取目标告警字段方法进行整体性的说明。
[0247] 图3是本申请实施例提供的一种提取目标告警字段方法的流程示意图。
[0248] 如 图3所示, 图3包括以下步骤S301~S312。
[0249] S301、获取原始告警JSON对象。
[0250] S302、获取内容提取规则LinkedHashMap对象。
[0251] S303、规则类型判断。
[0252] S304、判断属于keywords类型。
[0253] S305、判断规则中是否有key在告警中。
[0254] S306、从告警json中提取出规则第一个被包含的key对应内容并返回。
[0255] S307、返回"null”。
[0256] S308、判断属于Condition类型。
[0257] S309、告警匹配方式判别。
[0258] S310、返回告警json指定字段值和内容提取规则的正则表达式的正则匹配结果。
[0259] S311、返回告警json对象和内容提取规则keywords列表第一个匹配的key对应的值。
[0260] S312、返回由内容提取规则指定key的告警json中对应的json的某个key对应的值。
[0261] 在一种实现方式中,利用标准告警格式对目标告警数据进行标准化处理,得到标准告警数据,标准告警格式包括以下至少一项:告警时间信息、告警严重程度信息、触发告警的设备类型信息;根据标准告警数据生成标准告警结果。
[0262] 标准告警格式是一个预定义的结构,描述了告警的基本信息和处理指令,包括一些关键字段,例如:告警时间信息、告警严重程度信息、触发告警的设备类型信息等等,具体的内容可以自定义设置,此处不作限定。
[0263] 也可以理解为,标准告警格式是通过分析各种告警数据后提炼出的规范,用于统一不同设备、类型、来源的告警信息,确保一致性和可读性。
[0264] 目标告警数据的标准化处理是将来源不同的告警数据经过转换、清洗、分类,最终映射到标准告警格式的过程。例如,进行码值转换处理、分类赋值处理等等。
[0265] 码值转换处理:将告警级别、设备类型等字段从原始数据的表示形式(如字符串、数字编码)转换为标准格式。这可以使用查找表(mapping table)或字典(dictionary)进行转换。
[0266] 分类赋值处理:对告警信息进行分类,给每个告警分配适当的标准化值,确保每个告警都符合标准告警格式。
[0267] 标准告警结果是基于标准告警数据生成的最终输出,可以理解为是对用户或系统的一次警告反馈,包含告警的主要信息及后续处理建议。例如,可以从标准告警数据中提取关键信息(如时间、告警级别、信息说明),根据告警级别和处理策略生成建议的处理措施。
[0268] 对目标告警数据进行标准化处理可以确保告警信息的一致性,便于后续的数据处理和自动化响应。同时,能够提高信息传递的清晰度,减少理解上的歧义。
[0269] 下面结合 图4来对上文中提到的告警数据处理方法进行整体性的说明。
[0270] 图4是本申请实施例提供的一种告警数据处理方法的流程示意图。
[0271] 如 图4所示, 图4包括以下步骤S401~S413。
[0272] S401、消息接入。
[0273] S402、数据加载。
[0274] S403、内容解析。
[0275] 上述步骤可以理解为是通过flink从kafka获取第三方探针告警数据,并解析为原始告警数据流。
[0276] S404、文本加载。
[0277] 更新yaml配置文件,增加对原始告警数据内容的提取逻辑,也可以理解为是加载内容提取配置文件(预设告警数据提取规则配置文件)的信息。
[0278] S405、条件解析。
[0279] S406、状态广播。
[0280] flink读取并解析yaml配置文件,将提取逻辑以广播流形式发布。
[0281] S407、连接流操作。
[0282] 连接原始告警数据流与广播流,生成告警配置连接流。
[0283] S408、条件匹配。
[0284] S409、内容提取。
[0285] S410、告警处理中间结果。
[0286] 在告警配置连接流中,遍历配置流条件进行匹配,成功匹配的条件用于从告警数据提取内容,生成中间告警处理结果。
[0287] S411、码值转换。
[0288] S412、分类赋值。
[0289] S413、标准告警结果推送。
[0290] 根据标准告警格式和 / 或第三方文档,进一步处理中间结果,形成并推送标准告警结果。
[0291] 下面结合具体的示例对 图3中的步骤进行说明。
[0292] 首先介绍一种可能的yaml配置文件的具体代码,如下所示:
[0293] {_label:
[0294] keywords:
[0295] -null
[0296] _status:keywords:
[0297] null
[0298] _caseid:keywords:
[0299] null
[0300] tags:
[0301] keywords:
[0302] -null
[0303] srCTP:
[0304] selection_1:
[0305] keywords|contain:
[0306] src_ip
[0307] sender_ipcondition:selection_1
[0308] srcPort:
[0309] selection_1:
[0310] keywords|contain:
[0311] -src_portcondition:selection_1destIP:
[0312] selection_1:
[0313] keywords|contain:
[0314] -dst_ipcondition:selection_1destPort:
[0315] selection_1:
[0316] keywords:-dst_portcondition:selection_1destMAC:
[0317] selection_1:
[0318] szResult|re:<dst_mac>(.*?)< / dst_mac>condition:selection_1name:
[0319] selection_1:
[0320] szResult|re:<sub_desc>(.*?)< / sub_desc>condition:selection_1
[0321] severity:
[0322] keywords:
[0323] -nLevel
[0324] success:
[0325] selection_1:
[0326] keywords|contain:
[0327] -success
[0328] condition:selection_1
[0329] time:
[0330] keywords:
[0331] -ts
[0332] -time
[0333] alertType:
[0334] keywords:
[0335] -nAttackType
[0336] source:
[0337] keywords:
[0338] -logtype
[0339] rawLog:
[0340] keywords:
[0341] null}。
[0342] 通过flink从kafka获取第三方探针告警数据,并解析为原始告警数据流,这里以上文中提到的告警对象alarmJson作为告警数据样例。
[0343] 更新yaml配置文件,增加对原始告警数据内容的提取逻辑,这里使用上文中的yaml配置文件。
[0344] flink读取并解析yaml配置文件,将提取逻辑以广播流形式发布。值得注意的是,这里广播是一种逻辑概念,本质还是yaml配置文件。
[0345] 连接原始告警数据流与广播流,生成告警配置连接流。这里告警配置连接流同样是逻辑概念,仍用告警对象alarmJson和yaml配置文件进行说明。
[0346] 在告警配置连接流中,遍历配置流条件进行匹配,成功匹配的条件用于从告警数据提取内容,生成中间告警处理结果。用代码表示如下:
[0347] {"destIP":"157.245.134.176",
[0348] "severity":2,
[0349] "srcIP":"100.0.8.147",
[0350] "alertType":3,
[0351] "_caseid":"null",
[0352] "rawlog";"null",
[0353] "srcPort":51399,
[0354] "source":"utm",
[0355] "_status";"null",
[0356] "_label":"null",
[0357] "tags":"null",
[0358] "destPort";443,
[0359] "success":"null",
[0360] "name":"发现病毒:Win32 / Trojan.Gen.PK",
[0361] "srcMAC":"04:7c:16:d0:e1:80",
[0362] "time":1721935130,
[0363] "destMAC":"14:51:7:6:96:01"}。
[0364] 对告警处理中间结果再次根据标准告警格式及第三方对接文档进行处理,形成最终的标准告警结果,用代码表示如下:
[0365] {"destIP":"60.163.172.90”,
[0366] "severity":2,
[0367] "srcIP":"100.0.10.166",
[0368] "alertType":"配置错误漏洞",
[0369] "_caseid":"",
[0370] "rawLog":"",
[0371] "srcPort":52306,
[0372] "source":"JGKJ_W",
[0373] "_status":0,
[0374] "_label":"Alert",
[0375] "tags":[],
[0376] "destPort":80,
[0377] "success":0,
[0378] "name":"配置错误漏洞”,
[0379] "srcMAC":"null",
[0380] "time":1721890394000,
[0381] "destMAC":"null"}。
[0382] 具体的,将告警处理中间结果的业务字段"_caseId”、"status”、"_label”以及"tags”分别用"”,"0”"Alert”和"[]”进行填充。告警类型字段alertType根据对接文档进行码值转换将""alertType":3”中的3改为"配置错误漏洞”。
[0383] 结合上文中提到的告警数据处理方法,通过预先在yaml配置文件中设定的两种预设告警数据提取规则,可准确提取各种第三方告警数据,并且原始告警数据的处理流程采用flink框架,在实时计算场景中特别高效,确保了高实时性。
[0384] 换句话说,通过修改yaml配置文件,而非定制化开发,就可以实现对第三方告警数据的标准化处理。这样,几乎无需调整代码就能通过"一套代码+一个配置文件”处理多种告警数据,相比于传统方案中的针对每一种不同的告警数据开发一套不同的代码,本申请的方法可以大幅度降低开发成本并提升部署交付效率,能够实现告警数据的实时高效处理。
[0385] 上文主要结合附图对本申请实施例的方法进行了介绍。应理解,虽然如上所述的各实施例所涉及的流程图中的各个步骤依次显示,但是这些步骤并不是必然按照图中所示的顺序依次执行。除非本文中有明确的说明,这些步骤的执行并没有严格的顺序限制,这些步骤可以以其它的顺序执行。而且,如上所述的各实施例所涉及的流程图中的至少一部分步骤可以包括多个步骤或者多个阶段,这些步骤或者阶段并不必然是在同一时刻执行完成,而是可以在不同的时刻执行,这些步骤或者阶段的执行顺序也不必然是依次进行,而是可以与其它步骤或者其它步骤中的步骤或者阶段的至少一部分轮流或者交替地执行。下面结合附图对本申请实施例的一种的装置进行介绍。为了简洁,在下文介绍装置时,会进行适当省略,相关内容可以参照上文的方法中的相关描述,不再重复介绍。
[0386] 图5是申请实施例提供的一种告警数据处理装置的结构示意图。
[0387] 如 图5所示,该装置1000包括以下单元。
[0388] 获取单元1001,用于获取原始告警数据流和预设告警数据提取规则,预设告警数据提取规则是通过在配置文件中配置得到的;
[0389] 处理单元1002,用于对预设告警数据提取规则进行解析,得到告警数据提取规则广播流;
[0390] 处理单元1002还用于,对原始告警数据流和告警数据提取规则广播流进行连接处理,得到告警配置连接流;
[0391] 处理单元1002还用于,从告警配置连接流中筛选出目标告警数据,目标告警数据为满足预设告警数据提取规则的告警数据。
[0392] 在一种实现方式中,该装置1000还包括存储单元1003,该存储单元1003可以用于存储指令和 / 或数据,从而实现上述实施例中的方法。
[0393] 在一种实现方式中,处理单元1002还用于,检测预设告警数据提取规则的标识信息,标识信息包括关键值标识信息和 / 或条件标识信息;当检测到关键值标识信息时,将预设告警数据提取规则确定为关键值提取规则;或者,当检测到条件标识信息时,将预设告警数据提取规则确定为条件提取规则。
[0394] 通过检测预设告警数据提取规则的标识信息,系统能够自动识别出告警数据提取的类型(关键值提取规则或条件提取规则)。这种自动识别能力提高了系统的灵活性和适应性,使得可以根据不同的告警数据结构和需求进行相应的处理。
[0395] 在一种实现方式中,处理单元1002还用于,当告警数据提取规则广播流是对关键值提取规则进行连接处理得到的时,从告警配置连接流中筛选出目标告警数据,包括:获取在配置文件中配置的目标关键值;根据目标关键值在告警配置连接流中进行遍历查找,将关键值与目标关键值相等的告警数据确定为目标告警数据。
[0396] 当告警数据提取规则是关键值提取规则时,系统能够通过预设的目标关键值在告警配置连接流中进行高效查找。这种方法确保了目标告警数据的准确性和相关性,避免了无关数据的干扰。通过将关键值与目标关键值进行比较,系统能够快速定位到与特定事件或条件相关的告警信息,便于后续处理。
[0397] 在一种实现方式中,处理单元1002还用于,从目标告警数据中提取与目标关键值相等的字段,得到目标告警字段;根据目标告警字段生成目标告警结果,目标告警结果用于指示用户执行告警操作。
[0398] 通过提取与目标关键值相等的字段,形成目标告警字段,有助于系统专注于与告警相关的重要信息。生成的目标告警结果可以清晰地指示用户需要采取的操作,提升用户的响应效率。
[0399] 在一种实现方式中,处理单元1002还用于,当告警数据提取规则广播流是对条件提取规则进行连接处理得到的时,从告警配置连接流中筛选出目标告警数据,包括:获取在配置文件中配置的目标条件值,目标条件值是通过以下至少一项匹配方式得到的:正则表达式匹配方式、包含匹配方式和JSON匹配方式;根据目标条件值在告警配置连接流中进行遍历查找,将满足目标条件值的告警数据确定为目标告警数据。
[0400] 当告警数据提取规则是条件提取规则时,系统通过多种匹配方式(如正则表达式、包含匹配、JSON匹配)来筛选目标告警数据。这种灵活的匹配方式使得系统可以处理更复杂的告警条件,适应不同数据格式和结构的告警信息。通过条件提取,能够捕捉到更广泛的告警信息,确保不遗漏重要的告警事件。
[0401] 在一种实现方式中,处理单元1002还用于,从目标告警数据中提取满足正则表达式匹配方式的字段,得到目标告警字段;或者,从目标告警数据中提取满足包含匹配方式的字段,得到目标告警字段;或者,从目标告警数据中提取满足JSON匹配方式的字段,得到目标告警字段;根据目标告警字段生成目标告警结果,目标告警结果用于指示用户执行告警操作。
[0402] 根据不同的匹配方式(正则表达式、包含匹配、JSON匹配)提取目标告警字段,确保系统能够获取到符合特定条件的告警信息。这种多样化的提取方式增强了系统的灵活性,使得能够应对不同的告警场景和需求。生成的目标告警结果为用户提供了明确的操作指示,提升了告警处理的效率。
[0403] 在一种实现方式中,处理单元1002还用于,利用标准告警格式对目标告警数据进行标准化处理,得到标准告警数据,标准告警格式包括以下至少一项:告警时间信息、告警严重程度信息、触发告警的设备类型信息;根据标准告警数据生成标准告警结果。
[0404] 将目标告警数据进行标准化处理,确保告警信息的一致性和可读性,便于后续的数据管理和分析。标准告警格式的使用(如告警时间、严重程度、设备类型等)使得告警信息更加结构化,便于系统和用户理解。生成的标准告警结果能够以统一的格式反馈给用户,提升告警处理的专业性和效率,同时也为后续的自动化处理和数据分析打下基础。
[0405] 需要说明的是,上述单元之间的信息交互、执行过程等内容,由于与本申请方法实施例基于同一构思,其具体功能及带来的技术效果,具体可参见方法实施例部分,此处不再赘述。
[0406] 图6是本申请实施例提供的计算机设备的结构示意图。如 图6所示,该实施例的计算机设备3000包括:至少一个处理器3100( 图6仅示出一个)处理器、存储器3200以及存储在存储器3200中并可在至少一个处理器3100上运行的计算机程序3210,处理器3100执行计算机程序3210时,使得所述计算机设备实现上述实施例中的步骤。
[0407] 处理器3100可以是中央处理单元(Central Processing Unit,CPU),该处理器3100还可以是其他通用处理器、数字信号处理器(Digital Signal Processor,DSP)、专用集成电路(Application Specific Integrated Circuit,ASIC)、现成可编程门阵列(Field-Programmable Gate Array,FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
[0408] 存储器3200在一些实施例中可以是计算机设备3000的内部存储单元,例如计算机设备3000的硬盘或内存。存储器3200在另一些实施例中也可以是计算机设备3000的外部存储设备,例如计算机设备3000上配备的插接式硬盘,智能存储卡(Smart Media Card,SMC),安全数字(Secure Digital,SD)卡,闪存卡(Flash Card)等。进一步地,存储器3200还可以既包括计算机设备3000的内部存储单元也包括外部存储设备。存储器3200用于存储操作系统、应用程序、引导装载程序(Boot Loader)数据以及其他程序等,例如计算机程序的程序代码等。存储器3200还可以用于暂时地存储已经输出或者将要输出的数据。
[0409] 所属领域的技术人员可以清楚地了解到,为了描述的方便和简洁,仅以上述各功能单元的划分进行举例说明,实际应用中,可以根据需要而将上述功能分配由不同的功能单元、模块完成,即将所述装置的内部结构划分成不同的功能单元或模块,以完成以上描述的全部或者部分功能。实施例中的各功能单元可以集成在一个处理单元中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个单元中,上述集成的单元既可以采用硬件的形式实现,也可以采用软件功能单元的形式实现。另外,各功能单元的具体名称也只是为了便于相互区分,并不用于限制本申请的保护范围。上述系统中单元的具体工作过程,可以参考前述方法实施例中的对应过程,在此不再赘述。
[0410] 本申请实施例还提供了一种计算机可读存储介质,计算机可读存储介质存储有计算机程序,计算机程序被计算机设备执行时,使得计算机设备实现可实现上述各个方法实施例中的步骤。
[0411] 本申请实施例提供了一种计算机程序产品,当计算机程序产品在计算机设备上运行时,使得计算机设备能够实现上述各个方法。
[0412] 所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时,可以存储在一个计算机可读取存储介质中。基于这样的理解,本申请实现上述实施例方法中的全部或部分流程,可以通过计算机程序来指令相关的硬件来完成,所述的计算机程序可存储于一计算机可读存储介质中,该计算机程序在被处理器执行时,使得计算机设备可实现上述各个方法实施例的步骤。其中,所述计算机程序包括计算机程序代码,所述计算机程序代码可以为源代码形式、对象代码形式、可执行文件或某些中间形式等。所述计算机可读介质至少可以包括:能够将计算机程序代码携带到拍照装置 / 终端设备的任何实体或装置、记录介质、计算机存储器、只读存储器(Read-Only Memory,ROM)、随机存取存储器(Random Access Memory,RAM)、电载波信号、电信信号以及软件分发介质。例如U盘、移动硬盘、磁碟或者光盘等。在某些司法管辖区,根据立法和专利实践,计算机可读介质不可以是电载波信号和电信信号。
[0413] 应理解,上述实施例中各步骤的序号的大小并不意味着执行顺序的先后,各过程的执行顺序应以其功能和内在逻辑确定,而不应对本申请实施例的实施过程构成任何限定。在描述中,为了说明而不是为了限定,提出了诸如特定系统结构、技术之类的具体细节,以便透彻理解本申请实施例。然而,本领域的技术人员应当清楚,在没有这些具体细节的其它实施例中也可以实现本申请。在其它情况中,省略对众所周知的系统、装置、电路以及方法的详细说明,以免不必要的细节妨碍本申请的描述。
[0414] 应当理解,当在本申请说明书和所附权利要求书中使用时,术语"包括”指示所描述特征、整体、步骤、操作、元素和 / 或组件的存在,但并不排除一个或多个其它特征、整体、步骤、操作、元素、组件和 / 或其集合的存在或添加。
[0415] 还应当理解,在本申请说明书和所附权利要求书中使用的术语"和 / 或”是指相关联列出的项中的一个或多个的任何组合以及所有可能组合,并且包括这些组合。
[0416] 另外,在本申请说明书和所附权利要求书的描述中,术语"包括”、"包含”、"具有”及它们的变形都意味着"包括但不限于”,除非是以其他方式另外特别强调。
[0417] 在上述实施例中,对各个实施例的描述都各有侧重,某个实施例中没有详述或记载的部分,可以参见其它实施例的相关描述。
[0418] 本领域普通技术人员可以意识到,结合本文中所公开的实施例描述的各示例的单元及算法步骤,能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行,取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能,但是这种实现不应认为超出本申请的范围。
[0419] 在本申请所提供的实施例中,应该理解到,所揭露的装置、计算机设备和方法,可以通过其它的方式实现。例如,以上所描述的装置、计算机设备实施例仅仅是示意性的,例如,单元的划分,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式,例如多个单元或组件可以结合或者可以集成到另一个系统,或一些特征可以忽略,或不执行。另一点,所显示或讨论的相互之间的耦合或直接耦合或通讯连接可以是通过一些接口,装置或单元的间接耦合或通讯连接,可以是电性,机械或其它的形式。
[0420] 以上所述实施例仅用以说明本申请的技术方案,而非对其限制;尽管参照前述实施例对本申请进行了详细的说明,本领域的普通技术人员应当理解:其依然可以对前述各实施例所记载的技术方案进行修改,或者对其中部分技术特征进行等同替换;而这些修改或者替换,并不使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围,均应包含在本申请的保护范围之内。 < / url> < / root> < / url> < / root>
Claims
1. A method for processing alarm data, characterized in that: include: Acquire the original alarm data stream and the preset alarm data extraction rule, wherein the preset alarm data extraction rule is obtained by configuring in the configuration file; Parsing the preset alarm data extraction rule to obtain an alarm data extraction rule broadcast stream; Connecting the original alarm data stream and the alarm data extraction rule broadcast stream to obtain an alarm configuration connection stream; Target alarm data is screened out from the alarm configuration connection flow, where the target alarm data is alarm data that meets the preset alarm data extraction rule.
2. The method according to claim 1, characterized in that The preset alarm data extraction rule includes a key value extraction rule and / or a conditional extraction rule, and the method further includes: Detecting identification information of the preset alarm data extraction rule, the identification information including key value identification information and / or condition identification information; When the key value identification information is detected, the preset alarm data extraction rule is determined as the key value extraction rule; or, When the condition identification information is detected, the preset alarm data extraction rule is determined as the condition extraction rule.
3. The method according to claim 2, characterized in that When the alarm data extraction rule broadcast stream is obtained by performing connection processing on the key value extraction rule, the step of filtering out target alarm data from the alarm configuration connection stream includes: Get the target key value configured in the configuration file; A traversal search is performed in the alarm configuration connection flow according to the target key value, and the alarm data having a key value equal to the target key value is determined as the target alarm data.
4. The method according to claim 3, characterized in that The method further comprises: Extracting a field equal to the target key value from the target alarm data to obtain a target alarm field; A target alarm result is generated according to the target alarm field, and the target alarm result is used to instruct a user to perform an alarm operation.
5. The method according to claim 2, characterized in that: When the alarm data extraction rule broadcast stream is obtained by connecting and processing the conditional extraction rule, the step of filtering out target alarm data from the alarm configuration connection stream includes: Obtaining a target condition value configured in the configuration file, wherein the target condition value is obtained by at least one of the following matching methods: a regular expression matching method, an inclusion matching method, and a JSON matching method; The alarm configuration connection flow is traversed and searched according to the target condition value, and the alarm data satisfying the target condition value is determined as the target alarm data.
6. The method according to claim 5, characterized in that The method further comprises: Extracting fields satisfying the regular expression matching method from the target alarm data to obtain target alarm fields; or, Extracting fields satisfying the inclusion matching method from the target alarm data to obtain target alarm fields; or, Extracting fields satisfying the JSON matching method from the target alarm data to obtain target alarm fields; A target alarm result is generated according to the target alarm field, and the target alarm result is used to instruct a user to perform an alarm operation.
7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: The target alarm data is standardized by using a standard alarm format to obtain standard alarm data, wherein the standard alarm format includes at least one of the following: alarm time information, alarm severity information, and alarm triggering device type information; A standard alarm result is generated according to the standard alarm data.
8. An alarm data processing device, characterized in that: include: An acquisition unit, used to acquire an original alarm data stream and a preset alarm data extraction rule, wherein the preset alarm data extraction rule is obtained by configuring in a configuration file; A processing unit, configured to parse the preset alarm data extraction rule to obtain an alarm data extraction rule broadcast stream; The processing unit is further used to connect the original alarm data stream and the alarm data extraction rule broadcast stream to obtain an alarm configuration connection stream; The processing unit is further configured to filter out target alarm data from the alarm configuration connection flow, where the target alarm data is alarm data that satisfies the preset alarm data extraction rule.
9. A computer device, characterized in that: The computer device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the computer device implements the method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a computer device, the method according to any one of claims 1 to 7 is implemented.