A method for analyzing firewall log data

Abstract

The invention relates to the technical field of firewalls, and discloses a method for analyzing firewall log data, comprising the following steps: collecting firewall log data, the firewall log data including original network data packets, traffic statistical features, and log text; splitting the log text into words, and then identifying words belonging to events from the set of split words as an event word set; generating a graph structure data for each event word, sorting the graph structure data corresponding to all event words to obtain sequence data; inputting the sequence data into a log data analysis model, and outputting whether the event corresponding to the event word is the result of a dangerous event; the method of the invention can automatically learn feature patterns from a large amount of data without the need for artificial rules, thereby greatly improving the analysis efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of firewalls, and more specifically, to a method for parsing firewall log data. Background Art

[0002] The existing firewall log analysis methods have some shortcomings, mainly including:

[0003] Inefficiency: Traditional log analysis methods based on rules or pattern matching require a lot of manual intervention, and the analysis process is slow and error-prone. With the surge in network traffic and log data, manual analysis can no longer meet the needs of real-time response.

[0004] Unable to discover complex attack patterns: Some potential security threats may manifest as complex attack patterns. Traditional methods find it difficult to discover these hidden abnormal behaviors from massive log data.

[0005] Lack of correlation mining: A single log entry often cannot reflect the overall attack situation, and existing analysis methods lack the ability to perform correlation mining on different log entries. Summary of the invention

[0006] The invention provides a firewall log data parsing method, which solves the technical problems of firewall log analysis methods in related technologies.

[0007] The present invention provides a firewall log data parsing method, comprising the following steps:

[0008] Step 100, collecting firewall log data, the firewall log data includes original network data packets, traffic statistics characteristics, and log text;

[0009] Step 200, splitting the log text into words, and then identifying the words belonging to the event from the set of split words as an event word set;

[0010] Step 300, generating a graph structure data for each event word, the graph structure data including nodes and edges, the nodes including event words, user behaviors, servers and computers, if there is an edge between two nodes representing event words and user behaviors respectively, it means that the user behaviors caused the event of the event word; if there is an edge between two nodes representing event words and servers respectively, it means that the event of the event word occurred on the server; if there is an edge between two nodes representing event words and computers respectively, it means that the event of the event word affected the computer;

[0011] Sort the words in the event word set according to the order in the log text to obtain an event word sequence, and then sort the graph structure data corresponding to all event words to obtain sequence data, the sequence data includes n sequence units, and n represents the total number of words in the event word set;

[0012] Step 400, input the sequence data into the log data parsing model, the log data parsing model includes a first intermediate layer, a second intermediate layer and a first output layer, wherein the first intermediate layer inputs the sequence data, outputs the first hidden state to the second intermediate layer, the second intermediate layer outputs the second hidden state to the first output layer, and the first output layer outputs whether the event corresponding to the event word is the result of a dangerous event.

[0013] Furthermore, a method for identifying words belonging to an event is to pass the words into a word segmenter for identification.

[0014] Furthermore, firewall logs include system logs, configuration logs, status logs, alarm logs, and audit logs.

[0015] Furthermore, the calculation formula of the first intermediate layer of the log data parsing model is as follows:

[0016] ;

[0017] in Represents the first hidden state of the vth node of the graph structure data, is the set of nodes that have edges with the vth node of the graph structure data. Represents the input features of the u-th node of the graph structure data, is a normalizing constant, is the sigmoid function, represents the first weight parameter, is the first bias parameter.

[0018] Further, The default value is 10.

[0019] Furthermore, the calculation formula of the second intermediate layer is as follows:

[0020]

[0021] in represents the t-th second hidden state of the v-th node output by the second intermediate layer, represents the t-1th second hidden state of the vth node, It represents the first hidden state of the vth node output when the tth sequence unit of the sequence data is input into the first intermediate layer. and are the third and fourth weight parameters, is the second bias parameter and tanh is the hyperbolic tangent function.

[0022] Furthermore, the calculation formula of the first output layer is as follows:

[0023]

[0024] in represents the probability value that the event represented by the vth node is a dangerous event, represents the t-th second hidden state of the v-th node output by the second intermediate layer, is the first output weight parameter, is the first output bias parameter, Represents the sigmoid function.

[0025] Furthermore, the method also includes the step of sending information about dangerous events to operation and maintenance personnel.

[0026] Furthermore, the method also includes the step of revoking the authority of the user corresponding to the dangerous event.

[0027] The present invention provides a computer storage medium, which is used to store computer-readable instructions. When the computer-readable instructions are read, the aforementioned firewall log data parsing method can be executed.

[0028] The beneficial effects of the present invention are:

[0029] The method of the present invention can automatically learn characteristic patterns from a large amount of data without the need for manual rules, thus greatly improving the efficiency of analysis;

[0030] The model of the present invention can automatically capture the temporal correlation between logs, restore the overall attack process, and reduce the probability of analysis errors.

[0031] The present invention adopts deep learning technology, has autonomous learning and transfer learning capabilities, and can adapt to new logs and attack forms. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 The present invention is a flowchart of a method for analyzing firewall log data. DETAILED DESCRIPTION

[0033] The subject matter described herein will now be discussed with reference to example implementations. It should be understood that the discussion of these implementations is only to enable those skilled in the art to better understand and implement the subject matter described herein, and the functions and arrangements of the elements discussed may be changed without departing from the scope of protection of the present specification. Various examples may omit, replace, or add various processes or components as needed. In addition, the features described in some examples may also be combined in other examples.

[0034] At least one embodiment of the present invention discloses a method for parsing firewall log data. Figure 1 As shown, the following steps are included:

[0035] Step 100, collecting firewall log data, the firewall log data includes original network data packets, traffic statistics characteristics, and log text;

[0036] Step 200, splitting the log text into words, and then identifying the words belonging to the event from the set of split words as an event word set;

[0037] The method to identify words belonging to an event is to pass the words into a tokenizer for identification.

[0038] Firewall logs generally use a standardized format for easy reading and analysis. Common log formats include: System logs: record firewall system-level events, such as startup, shutdown, and restart.

[0039] Configuration log: records changes in firewall configuration information, such as the addition, deletion, and modification of policies and rules. Status log: records the working status of each module of the firewall, such as connection status, packet processing status, etc. Alarm log: records security events detected by the firewall, such as intrusion attempts, malware, port scanning, etc. Audit log: records access and operation records of the firewall, such as login, login failure, command execution, etc.

[0040] Step 300, generating a graph structure data for each event word, the graph structure data including nodes and edges, the nodes including event words, user behaviors, servers and computers, if there is an edge between two nodes representing event words and user behaviors respectively, it means that the user behaviors caused the event of the event word; if there is an edge between two nodes representing event words and servers respectively, it means that the event of the event word occurred on the server; if there is an edge between two nodes representing event words and computers respectively, it means that the event of the event word affected the computer;

[0041] Sort the words in the event word set according to the order in the log text to obtain an event word sequence, and then sort the graph structure data corresponding to all event words to obtain sequence data, the sequence data includes n sequence units, and n represents the total number of words in the event word set;

[0042] Step 400, input the sequence data into the log data parsing model, the log data parsing model includes a first intermediate layer, a second intermediate layer and a first output layer, wherein the first intermediate layer inputs the sequence data, outputs the first hidden state to the second intermediate layer, the second intermediate layer outputs the second hidden state to the first output layer, and the first output layer outputs whether the event corresponding to the event word is the result of a dangerous event.

[0043] The calculation formula of the first intermediate layer of the log data parsing model is as follows:

[0044] ;

[0045] in Represents the first hidden state of the vth node of the graph structure data, is the set of nodes that have edges with the vth node of the graph structure data. Represents the input features of the u-th node of the graph structure data, is a normalization constant (default value is 10), is the sigmoid function, represents the first weight parameter, is the first bias parameter.

[0046] In one embodiment of the present invention, the calculation formula of the second intermediate layer is as follows:

[0047]

[0048] in represents the t-th second hidden state of the v-th node output by the second intermediate layer, represents the t-1th second hidden state of the vth node, It represents the first hidden state of the vth node output when the tth sequence unit of the sequence data is input into the first intermediate layer. and are the third and fourth weight parameters, is the second bias parameter and tanh is the hyperbolic tangent function.

[0049] In one embodiment of the present invention, the calculation formula of the first output layer is as follows:

[0050]

[0051] in represents the probability value that the event represented by the vth node is a dangerous event, represents the t-th second hidden state of the v-th node output by the second intermediate layer, is the first output weight parameter, is the first output bias parameter, Represents the sigmoid function.

[0052] In one embodiment of the present invention, a firewall log data parsing method is provided, which also includes the step of sending information about dangerous events to operation and maintenance personnel.

[0053] In one embodiment of the present invention, a firewall log data parsing method is provided, which also includes the step of revoking the authority of the user corresponding to the dangerous event.

[0054] The present invention provides a computer storage medium, which is used to store computer-readable instructions. When the computer-readable instructions are read, the aforementioned firewall log data parsing method can be executed.

[0055] The above describes an embodiment of the present invention, but this embodiment is not limited to the above-mentioned specific implementation mode. The above-mentioned specific implementation mode is merely illustrative and not restrictive. Under the guidance of this embodiment, ordinary technicians in this field can also make more forms of equivalent embodiments, all of which are within the protection of this embodiment.

Claims

1. A method for analyzing firewall log data, characterized in that: The following steps are involved: Step 100, collecting firewall log data, the firewall log data includes original network data packets, traffic statistics characteristics, and log text; Step 200, splitting the log text into words, and then identifying the words belonging to the event from the set of split words as an event word set; Step 300, generating a graph structure data for each event word, the graph structure data including nodes and edges, the nodes including event words, user behaviors, servers and computers, if there is an edge between two nodes representing event words and user behaviors respectively, it means that the user behaviors caused the event of the event word; if there is an edge between two nodes representing event words and servers respectively, it means that the event of the event word occurred on the server; if there is an edge between two nodes representing event words and computers respectively, it means that the event of the event word affected the computer; Sort the words in the event word set according to the order in the log text to obtain an event word sequence, and then sort the graph structure data corresponding to all event words to obtain sequence data, the sequence data includes n sequence units, and n represents the total number of words in the event word set; Step 400, inputting the sequence data into a log data parsing model, the log data parsing model comprising a first intermediate layer, a second intermediate layer and a first output layer, wherein the first intermediate layer inputs the sequence data, outputs a first hidden state to the second intermediate layer, the second intermediate layer outputs a second hidden state to the first output layer, and the first output layer outputs whether the event corresponding to the event word is a result of a dangerous event; The calculation formula of the first intermediate layer of the log data parsing model is as follows: ； in Represents the first hidden state of the vth node of the graph structure data, is the set of nodes that have edges with the vth node of the graph structure data. Represents the input features of the u-th node of the graph structure data, is a normalizing constant, is the sigmoid function, represents the first weight parameter, is the first bias parameter; The calculation formula for the second intermediate layer is as follows: ； in represents the t-th second hidden state of the v-th node output by the second intermediate layer, represents the t-1th second hidden state of the vth node, It represents the first hidden state of the vth node output when the tth sequence unit of the sequence data is input into the first intermediate layer. and are the third and fourth weight parameters, is the second bias parameter, tanh is the hyperbolic tangent function; The calculation formula of the first output layer is as follows: ； in represents the probability value that the event represented by the vth node is a dangerous event, represents the t-th second hidden state of the v-th node output by the second intermediate layer, is the first output weight parameter, is the first output bias parameter, Represents the sigmoid function.

2. A method for analyzing firewall log data according to claim 1, characterized in that: The method to identify words belonging to an event is to pass the words into a tokenizer for identification.

3. A method for analyzing firewall log data according to claim 1, characterized in that: Firewall logs include system logs, configuration logs, status logs, alarm logs, and audit logs.

4. A method for analyzing firewall log data according to claim 1, characterized in that: The default value is 10.

5. A method for analyzing firewall log data according to claim 1, characterized in that: It also includes the step of sending information about dangerous events to operation and maintenance personnel.

6. A method for analyzing firewall log data according to claim 1, characterized in that: The method also includes the step of revoking the authority of the user corresponding to the dangerous event.

7. A computer storage medium, characterized in that: It is used to store computer-readable instructions, and when the computer-readable instructions are read, a firewall log data parsing method as described in any one of claims 1-6 can be executed.

Images