Network Security Incident Analysis and Processing Method, System, and Readable Storage Medium

Abstract

The present invention belongs to the field of network security technology, and specifically relates to a method, system, and readable storage medium for analyzing and processing network security events. The processing method includes: collecting the original alarm log information of security devices and performing generalization processing; calling an algorithm component library in the form of an interface for the data after generalization processing, and outputting security events after noise reduction analysis; wherein, the algorithm component library includes three algorithm modules: a bidirectional long short-term memory Bi-LSTM network module with an attention mechanism, a Prophet anomaly detection module based on time series analysis, and a collaborative filtering anomaly detection module based on a graph neural network GCN combined with spatial distance calculation; the three algorithm modules are synchronously called in parallel, and the final result takes the intersection to output its security events. The present invention uses a method for analyzing security events with different purposes by three algorithm modules to perform alarm noise reduction for security events from multiple dimensions and output important security events to improve the accuracy of alarms.

Description

Technical Field

[0001] The present invention belongs to the technical field of network security, and particularly relates to a method, a system and a readable storage medium for analyzing and processing network security events. Background Art

[0002] With the continuous development of network technology, network security issues have gradually become problems that need to be continuously updated and iteratively solved in the current technological development. In the prior art, based on the security alarm information sent by network security devices, relevant technical personnel need to manually view a large number of alarm messages and their corresponding logs, analyze their characteristics from the noise alarms, and then form the filtering rule logic of the noise alarms based on some characteristics, so as to achieve alarm noise reduction. However, how to improve the alarm efficiency and the accuracy of security event analysis on the premise of combining the current mainstream technologies has become a difficult problem that needs to be solved urgently. Summary of the Invention

[0003] Based on the above deficiencies existing in the prior art, the purpose of the present invention is to provide a method, a system and a readable storage medium for analyzing and processing network security events.

[0004] In order to achieve the above invention purpose, the present invention adopts the following technical solutions:

[0005] A method for analyzing and processing network security events includes the following steps:

[0006] S1. Collect the original alarm log information of security devices;

[0007] S2. Perform generalization processing on the original alarm log information;

[0008] S3. Call the algorithm component library in the form of an interface for the data after generalization processing, and output the security events after noise reduction analysis;

[0009] Among them, the algorithm component library includes three algorithm modules: a bidirectional long short-term memory Bi-LSTM network module with an attention mechanism, a Prophet anomaly detection module based on time series analysis, and a collaborative filtering anomaly detection module based on graph convolutional network GCN combined with spatial distance calculation; the three algorithm modules are synchronously called in parallel, and the final result is output by taking the intersection.

[0010] As a preferred solution, the original alarm log information includes source IP, destination IP, attack type, attack means, and attack payload Payload.

[0011] As a preferred solution, the Bi-LSTM network module includes a text data processing unit, an encoding unit of the attention mechanism, a Bi-LSTM network unit, and a decoding unit of the attention mechanism, which are connected in sequence, and the binary cross-entropy loss function BCELoss is used for iterative loop during the training process;

[0012] Input the generalized attack payload Payload into the bidirectional long short-term memory Bi-LSTM network module with an attention mechanism to output an alarm log S 1.

[0013] As a preferred solution, the detection process of the Prophet anomaly detection module includes:

[0014] Pass the generalized data through the Prophet time series anomaly analysis algorithm to output the alarm log value of the next time window; if the alarm log value is greater than the upper limit of the confidence interval of the alarm log quantity, it is determined as an alarm time series anomaly, and the fault tree analysis algorithm is used to perform root cause analysis on the time series window of the abnormal alarm, locate the abnormal time point or time window, and combine the Bayesian prior anomaly probability to output the posterior anomaly probability of the abnormal alarm of the same type of security device. When the posterior anomaly probability is greater than the target probability threshold, an alarm log is output S 2.

[0015] As a preferred solution, the detection process of the collaborative filtering anomaly detection module includes:

[0016] Construct a graph network chain based on the generalized data and store it in the graph database in the graph structure of node-line-node. Input the graph structure in the graph database into the GCN, and perform anomaly detection in combination with collaborative filtering based on spatial distance calculation to output the abnormal nodes and alarm logs between heterogeneous security devices S 3.

[0017] As a preferred solution, based on the alarm log S 1. Alarm log S 2. Alarm log S 3. Take the intersection S 1 ∩ S 2 ∩ S 3 is used as the output result of the algorithm component library, that is, the security event after noise reduction analysis.

[0018] As a preferred solution, the output form of the security event includes the original log of the security event, event analysis tags, and event danger levels.

[0019] The present invention also provides a network security event analysis and processing system, which applies the network security event analysis and processing method described in any one of the above solutions. The network security event analysis and processing system includes:

[0020] A collection module for collecting the original alarm log information of security devices;

[0021] A data processing module for generalizing the original alarm log information;

[0022] An algorithm component module for calling an algorithm component library in the form of an interface for the generalized data and outputting the security events after noise reduction analysis.

[0023] The present invention also provides a readable storage medium, in which instructions are stored. When the instructions run on a computer, the computer is made to execute the network security event analysis and processing method described in any one of the above solutions.

[0024] Compared with the prior art, the beneficial effects of the present invention are:

[0025] The present invention uses the Bi-LSTM network module in the algorithm component library to eliminate invalid alarms, duplicate alarms, and low-risk alarms, the Prophet anomaly detection module to screen the time-series anomaly alarms of similar devices, and the collaborative filtering anomaly detection module to screen the anomaly alarms with attack correlation relationships between different security devices, performs alarm noise reduction for security events from multiple dimensions, and finally takes the intersection to output security events, effectively improving the accuracy of alarms. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a flowchart of the network security event analysis and processing method according to Embodiment 1 of the present invention;

[0027] Figure 2 is a processing flowchart of the Bi-LSTM network module according to Embodiment 1 of the present invention;

[0028] Figure 3 is a processing flowchart of the Prophet anomaly detection module according to Embodiment 1 of the present invention;

[0029] Figure 4 is a processing flowchart of the collaborative filtering anomaly detection module according to Embodiment 1 of the present invention;

[0030] Figure 5 is a module architecture diagram of the network security event analysis and processing system according to Embodiment 1 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0031] To more clearly illustrate the embodiments of the present invention, the specific embodiments of the present invention will be described below with reference to the accompanying drawings. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts, and other embodiments can also be obtained.

[0032] Embodiment 1:

[0033] As Figure 1 shown, the network security event analysis and processing method of this embodiment includes the following processes:

[0034] (1) Data collection: Collect the original alarm log information of security devices;

[0035] Specifically, the original alarm log information includes source IP, destination IP, attack type, attack means, and attack payload;

[0036] Among them, the attack types include information leakage, vulnerability exploitation, denial-of-service (DoS) attack, malware, etc.; the attack means include remote scanning, brute force cracking, security bypass, botnet, risky access, Trojan horse, etc.

[0037] (2) Data generalization: Perform generalization processing on the original alarm log information;

[0038] The generalization rules filter the original alarm log information and perform field mapping to obtain the data to be reported.

[0039] Before configuring the rules, the downstream needs to provide the original log and the corresponding field information of the original log, and perform data mapping with reference to the field information that needs to be reported in the data specification. Specifically, the collected original log data is sent to the specified topic of the stream processing platform Kafka through the real-time data transmission engine Logstash program. The stream processing framework flink generalization program monitors this topic of Kafka, performs generalization analysis on the collected original log through various generalization rules, and outputs it to the distributed and scalable real-time search and analysis engine ElasticSearch through the Logstach program to obtain the data after generalization processing;

[0040] (3) The data after generalization processing is input into the algorithm component library, and three algorithm modules in the algorithm component library are called for parallel processing. The final result output takes the intersection of the processing results of the three algorithm modules as the output of the entire algorithm component library, that is, the security event after noise reduction analysis;

[0041] Among them, the output form of the security event includes the original log of the security event, the event analysis label (i.e., valid / invalid), and the event danger level (high / medium / low).

[0042] The algorithm component library of this embodiment includes three algorithm modules:

[0043] The first algorithm module: a bidirectional long short-term memory network module with an attention mechanism, abbreviated as the Bi-LSTM network module;

[0044] The second algorithm module: Prophet anomaly detection module based on time series analysis, hereinafter referred to as Prophet anomaly detection module;

[0045] The third algorithm module: collaborative filtering anomaly detection module based on graph convolutional network combined with spatial distance calculation, hereinafter referred to as collaborative filtering anomaly detection module combining GCN with spatial distance calculation.

[0046] The above-mentioned first algorithm module is based on text classification. After generalizing the data through a pre-trained Bi-LSTM network module, it outputs alarm classification labels, namely valid / invalid, and the threat levels of valid alarms are high / medium / low risk, and finally obtains alarm logs. S 1. The function of the first algorithm module in this embodiment is to eliminate invalid alarms, duplicate alarms, and low-risk alarms.

[0047] The Bi-LSTM network module in this embodiment includes a text data processing unit, an encoding unit with an attention mechanism, a bidirectional long short-term memory neural network unit, and a decoding unit with an attention mechanism connected in sequence, and uses the binary cross-entropy loss function BCELoss for iterative training during the training process. As Figure 2 shown, the attack payload first undergoes text data processing, then encoding with the attention mechanism, the encoded data is input into the bidirectional long short-term memory neural network, and finally the output is achieved through decoding with the attention mechanism; during the training process, the binary cross-entropy loss function BCELoss is used for iterative training between the encoding and decoding of the attention mechanism. Among them, the text data processing unit performs word segmentation and vectorization conversion on the text. Specifically, the pre-training process of the Bi-LSTM network module is as follows:

[0048] Based on the keyword field attack payload Payload text in the generalized log sample data collected in the past 24 hours, security expert data annotation is performed, and data samples with labels are collected for the pre-trained network. Its labels include whether it is valid (yes / no) and the danger level (high / medium / low); after data annotation, the data samples are used for model modeling training under the condition that the positive and negative sample ratios are close to 1:1. The model network structure is designed as a bidirectional long short-term memory neural network Bi-LSTM with an attention mechanism. Bi-LSTM is a special recurrent neural network that can process sequential data and maintain long-term memory. Different from traditional recurrent neural networks, Bi-LSTM considers both past and future information, enabling the model to better capture the context information of sequential data. Considering the time continuity and repeatability between alarm logs, more modeling is considered from the time dimension. Therefore, during data input, it is first transmitted to Bi-LSTM through the encoding process of the attention mechanism, and then output as the output of the entire network through the decoding process of the attention mechanism. The loss function BCELoss is selected for iterative training, and finally a model with the best training effect is output.

[0049] As Figure 3 shown, the above - mentioned second algorithm module is based on time - series anomaly detection. After generalizing the data, it uses the pre - trained Prophet time - series anomaly analysis algorithm to output the alarm log value of the next time window. Based on historical time - series data analysis, it calculates the upper and lower limits of the number of alarm logs and constructs a confidence interval for the number of alarm logs. When the actual alarm log value is greater than the upper limit of the confidence interval, it is determined as an alarm time - series anomaly. For the time window of the abnormal alarm, it uses the fault tree analysis algorithm for root - cause analysis, locates the abnormal time point or time window, and combines the Bayesian prior anomaly probability to output the posterior anomaly probability of the abnormal alarm of the same type of security devices. When the posterior anomaly probability is greater than the target probability threshold (such as 95%), it outputs the alarm log S 2. This embodiment outputs the abnormal alarms of the same type of security devices based on the root - cause analysis of the fault tree analysis algorithm and the Bayesian prior experience, realizing the screening of time - series abnormal alarms of the same type of security devices. The specific analysis processes of the above - mentioned fault tree analysis algorithm and the Bayesian prior experience can refer to the prior art and will not be elaborated here.

[0050] The training process of the pre - trained Prophet time - series anomaly analysis algorithm in this embodiment is as follows:

[0051] Taking a single security monitoring device as the main body, it aggregates data from the time - series dimension, summarizes the time - series data feature groups of the security device, and selects the open - source time - series model Prophet of Facebook. This is a time - series prediction based on an additive model, suitable for time - series with seasonal effects and historical data with multiple seasons. Combining historical data analysis, it is considered that the current security device has certain seasonal patterns, so the Prophet model is selected for training. The specific structure of the Prophet model can refer to the prior art and will not be elaborated here.

[0052] As Figure 4 shown, the processing process of the above - mentioned third algorithm module includes: building a security knowledge expert database, building a knowledge graph, that is, a graph network chain, based on the attack knowledge chain in the security knowledge expert database and the generalized log data, and storing it in the graph database in the form of a graph structure of node - line - node. On this basis, for the large amount of accumulated graph structures in the graph database, it inputs the pre - trained GCN combined with the collaborative filtering recommendation method based on spatial distance calculation for abnormal association determination, outputs the abnormal nodes between different types of security devices, and at the same time outputs the abnormal alarm logs S 3. It realizes the screening of abnormal alarms with attack - related relationships between different types of security devices.

[0053] The training process of the GCN combined with the collaborative filtering anomaly detection module based on spatial distance calculation in this embodiment is as follows:

[0054] The security knowledge graph network chain collected based on the expert library is input into the graph convolutional network GCN with collaborative filtering considering spatial distance calculation. When the graph data is first input into the GCN, after several iterations through the hidden layer, the ReLU activation function layer, and the Dropout layer, the intermediate layer of the iterated feature matrix is input into the spatial calculation layer based on the collaborative filtering algorithm. After matrix spatial calculation, it is connected to the fully connected network layer and the output network result is calculated through the Softmax classification function layer. The result includes whether the graph is abnormal and the abnormal nodes (i.e., alarm logs). As a network that can capture the complex relationships and features between nodes in a graph, the core idea of GCN is to update the representation of each node through the information of neighboring nodes.

[0055] Finally, based on the alarm logs output by the three algorithm modules in this embodiment S 1. Alarm log S 2. Alarm log S 3. Take their intersection S 1 ∩ S 2 ∩ S 3 is used as the output result of the entire algorithm component library, that is, the security events after noise reduction analysis.

[0056] Based on the above network security event analysis and processing method, as Figure 5 shown, the network security event analysis and processing system of this embodiment includes the following functional modules: a collection module, a data processing module, and an algorithm component module.

[0057] Specifically, the collection module of this embodiment is used to collect the original alarm log information of security devices;

[0058] The data processing module of this embodiment is used to perform generalization processing on the original alarm log information;

[0059] The algorithm component module of this embodiment is used to call the algorithm component library in the form of an interface for the data after generalization processing, and output the security events after noise reduction analysis;

[0060] The specific processing procedures of the above functional modules can adopt the detailed descriptions in the above network security event analysis and processing method, and will not be elaborated here.

[0061] In the computer-readable storage medium of this embodiment, instructions are stored. When the instructions run on a computer, the computer is made to execute the above network security event analysis and processing method, realizing the intelligent analysis and processing of network security events.

[0062] The above is only a detailed description of the preferred embodiments and principles of the present invention. For those of ordinary skill in the art, according to the idea provided by the present invention, there will be changes in the specific implementation manners, and these changes should also be regarded as the protection scope of the present invention.

Claims

1. A network security incident analysis and processing method, characterized in that: The following steps are involved: S1. Collect the original alarm log information of security equipment; S2. Generalize the original alarm log information; S3, call the algorithm component library in the form of an interface for the generalized data, and output the security events after noise reduction analysis; The algorithm component library includes three algorithm modules: a bidirectional long short-term memory Bi-LSTM network module with an attention mechanism, a Prophet anomaly detection module based on time series analysis, and a collaborative filtering anomaly detection module based on a graph convolutional network GCN combined with spatial distance calculation. The three algorithm modules are synchronously called in parallel, and the final result is the intersection output. The original alarm log information includes source IP, destination IP, attack type, attack method, and attack payload; The Bi-LSTM network module includes a text data processing unit, an encoding unit of an attention mechanism, a Bi-LSTM network unit, and a decoding unit of an attention mechanism connected in sequence, and uses a binary cross entropy loss function BCELoss for cyclic iteration during the training process; Input the generalized attack payload into the Bi-LSTM network module and output the alarm log S 1; The detection process of the Prophet anomaly detection module includes: The generalized data is passed through the Prophet timing anomaly analysis algorithm to output the alarm log value of the next time window; if the alarm log value is greater than the upper limit of the confidence interval of the number of alarm logs, it is determined to be an alarm timing anomaly, and the fault tree analysis algorithm is used to perform root cause analysis on the timing window of the abnormal alarm, locate the abnormal time point or time window, and combine the Bayesian prior abnormal probability to output the posterior abnormal probability of abnormal alarms of similar safety equipment. When the posterior abnormal probability is greater than the target probability threshold, the alarm log is output. S 2; The detection process of the collaborative filtering anomaly detection module includes: A graph network chain is constructed based on the generalized data and stored in the graph database in the form of a node-line-node graph structure. The graph structure in the graph database is input into GCN, and anomaly detection is performed in combination with collaborative filtering based on spatial distance calculation, and abnormal nodes and alarm logs between heterogeneous security devices are output. S 3; Based on alarm log S 1. Alarm log S 2. Alarm log S 3. Take the intersection as the output result of the algorithm component library, that is, the security event after noise reduction analysis; The output form of the security event includes the original log of the security event, the event analysis label and the event risk level.

2. A network security incident analysis and processing system, using the network security incident analysis and processing method according to claim 1, characterized in that: The network security incident analysis and processing system comprises: The collection module is used to collect the original alarm log information of the security equipment; Data processing module, used for generalizing the original alarm log information; The algorithm component module is used to call the algorithm component library in the form of an interface for the generalized data and output security events after noise reduction analysis.

3. A readable storage medium, wherein instructions are stored in the readable storage medium, characterized in that: When the instructions are executed on a computer, the computer is caused to execute the network security incident analysis and processing method as claimed in claim 1.

Images