Firewall log security detection method and system

Abstract

The application discloses a kind of firewall log security detection method and system, it is related to the technical field of network firewall, comprising: the original log record data obtained by collecting to the firewall to be detected;Original log record data is preprocessed to obtain triad structured data;Triad structured data is converted into numerical value and is spliced to obtain log feature vector;Once detection is carried out through deep support vector data description to obtain once detection result, generate preliminary abnormal log subset;Based on preliminary abnormal log subset, event trace construction obtains XES preliminary screening event log, and the workflow network to be detected is generated by inductive mining algorithm;Consistency check is calculated by edit distance algorithm to obtain secondary detection score, and secondary judgment obtains security detection result.The application can accurately identify hidden security threats, can capture complex abnormal behaviors of combination and serialization, balance high efficiency, high accuracy and interpretability, effectively reduce false alarm rate.

Description

Technical Field

[0001] This invention relates to the technical field of network firewalls, and in particular to a firewall log security detection method and system. Background Technology

[0002] In recent years, thanks to breakthroughs in big data, information and communication, and artificial intelligence technologies, firewall logs have evolved from simple traffic data recording devices to a means of detecting network security threats. Currently, cloud-edge collaboration is the mainstream model for distributed network deployment. Enterprise networks, cloud computing platforms, and IoT systems in this architecture exhibit massive traffic and heterogeneous device characteristics. Simultaneously, network attacks are becoming increasingly sophisticated, placing higher demands on the accuracy of firewall log security detection. In high-speed network environments, firewall logs are easily overwritten or lost due to insufficient storage space, leading to the failure of event tracing. Therefore, research on firewall log security detection methods that balance efficiency and accuracy has broad application prospects.

[0003] Currently, Chinese invention patent application CN202410255038.7 discloses a method for detecting abnormal behavior based on firewall packet filtering logs. This application includes: S10, constructing and quantifying the feature attributes of firewall packet filtering logs; S11, setting the feature attributes of firewall packet filtering logs; S12, setting abnormal behavior patterns; S13, filtering firewall packet filtering logs; S14, setting a time window; S15, performing statistical analysis on the selected firewall packet filtering logs for the relevant feature attributes, obtaining statistical values ​​for the relevant feature attributes; S20, abnormal behavior analysis: S21, selecting an unsupervised learning algorithm; S22, initializing the parameters of the unsupervised learning algorithm based on the statistical values ​​of the relevant feature attributes; S23, learning the statistical values ​​of the relevant feature attributes; S24, detecting abnormal behavior based on the learning results. This application can uncover potential abnormal behavior through firewall packet filtering logs. However, the anomalies detected by this application are specific to individual log records, and its accuracy in identifying a series of seemingly normal but abnormally combined covert attacks is not high. Summary of the Invention

[0004] The technical problem addressed by this invention is that existing firewall log detection technologies primarily analyze individual log records one by one. However, modern cybersecurity threats increasingly evade detection by employing seemingly normal but combined behaviors that can lead to attacks, such as Advanced Persistent Threats (APTs) and insider threats. Many existing firewall log detection technologies rely heavily on deep learning, resulting in opaque internal decision-making processes. They provide a threat category assessment in a black-box manner but fail to clearly explain why the log was deemed abnormal. This makes it difficult for security analysts to intuitively understand the flow, key points, and deviations of anomalous behavior. Furthermore, training complex models requires large amounts of high-quality labeled data, increasing costs.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0006] A firewall log security detection method includes:

[0007] Step S1: Collect raw log data from the firewall to be detected;

[0008] Step S2: Preprocess the original log data to obtain triplet structured data;

[0009] Step S3: Perform numerical transformation and vector concatenation on the triplet structured data to obtain the log feature vector;

[0010] Step S4: Perform a detection on the log feature vector using deep support vector data description to obtain a detection result and generate a preliminary subset of abnormal logs;

[0011] Step S5: Based on the preliminary abnormal log subset, construct event traces to obtain the XES preliminary screening event log, and generate the workflow network to be detected through inductive mining algorithm;

[0012] Step S6: The consistency check is performed using the edit distance algorithm to calculate the secondary detection score, and a secondary judgment is made to obtain the security detection result.

[0013] Preferably, step S1 involves collecting raw log data from the firewall under test based on the Syslog protocol, including:

[0014] Listen on the firewall's UDP port 514 or TCP port 6514 using the Syslog protocol to receive key-value log data from the Syslog message stream conforming to the RFC5424 standard.

[0015] Log data in key-value pair format includes timestamps, action types, session statistics, source IP, destination IP, source port, destination port, transport layer protocol number, service identifier, security response type, client type, policy ID, and geographic information;

[0016] Action types include allow, deny, client reset, and timeout;

[0017] Session statistics include the number of bytes sent, the number of bytes received, and the session duration;

[0018] Security response types include logging, alarm generation, and connection reset;

[0019] Client types include browsers, web crawlers, and API tools.

[0020] Preferably, step S21 involves parsing each record in the original log data using regular expressions to obtain a key-value pair dataset, including:

[0021] For numeric string values ​​and unquoted string values, the first capture instruction is executed to obtain the corresponding key-value pairs. The expression for the first capture instruction is:

[0022] ([^\\s]*)=([^"\\s]*)(\\s|$);

[0023] For string values ​​containing double quotes, the second capture instruction is executed to obtain the corresponding key-value pair. The expression for the second capture instruction is:

[0024] ([^\\s]*)="([^"]*)"(\\s|$);

[0025] Extract the timestamp field from the log records and convert it to the standardized ISO 8601 format;

[0026] Step S22: Based on the key-value pair dataset, fields are extracted and combined to obtain the triplet structured data corresponding to each log record. The expression for the triplet structured data is as follows:

[0027] [Identifier, Behavior Characteristic, Feature Characteristic];

[0028] The identifier includes the source IP, destination IP, source port, destination port, and transport layer protocol number;

[0029] The behavior identifier includes the action type, policy ID, and security response type;

[0030] The identifiers include session statistics, service identifier, client type, and geographic information.

[0031] Preferably, in step S31, the identifier undergoes a first-order numerical conversion, and the processing logic is as follows:

[0032] The source IP and destination IP in the identifier are encoded using IPv4 address to integer;

[0033] Perform outlier checks on the source and destination ports. If the integer value corresponding to the port does not exceed the range of 1 to 65535, directly retain the original integer value.

[0034] When the integer value corresponding to a port exceeds the range of 1 to 65535, the port will be uniformly mapped to 65536.

[0035] The transport layer protocol number is encoded using the IANA standard protocol number encoding, and the unknown protocol number is mapped to the value 0;

[0036] Step S32, perform a second numerical conversion on the behavior symbol, the processing logic includes:

[0037] The original integer value is directly retained for the policy ID in the behavior specification;

[0038] The action type and security response type are mapped and assigned values ​​using a preset first tag code. The mapping relationship is as follows:

[0039] Map allowed actions to 000, denied actions to 001, client reset actions to 002, and timeout actions to 003 in the action type;

[0040] Map the logging type in the security response category to 100, the alarm generation type to 101, and the connection reset type to 102.

[0041] Step S33, the third-order numerical transformation of the feature symbol, the processing logic is as follows:

[0042] Convert ISO 8601 format timestamps from string format to Unix format timestamps from integer format;

[0043] Z-score normalization is applied to the number of bytes sent, bytes received, and session duration in the session statistics.

[0044] The service identifier and client type are mapped and assigned using a preset second tag encoding. The mapping relationship is as follows:

[0045] Map HTTPS to 300, HTTP to 301, SSH to 302, FTP to 303, and other service identifiers to 304 in the service identifiers;

[0046] Map browsers to 400, web crawlers to 401, API tools to 402, and other client types to 403.

[0047] Geographic information is encoded and mapped according to the alpha-2 format of the ISO 3166-1 standard;

[0048] Step S34: The triplet structured data that has undergone numerical transformation is concatenated into vectors according to its original arrangement order to obtain a one-dimensional numerical log feature vector.

[0049] Preferably, in step S41, the log feature vector is received through an input layer containing 15 neurons, and each element in the log feature vector is fed into a corresponding neuron in the input layer.

[0050] The log feature vector is processed by a first linear transformation and a ReLU function nonlinear activation to obtain the output vector of the first hidden layer. The first hidden layer contains 32 neurons.

[0051] The output vector of the first hidden layer is used as the input data of the second hidden layer. The second linear transformation and ReLU nonlinear activation are performed to complete forward propagation and obtain the deep feature vector. The second hidden layer contains 16 neurons.

[0052] The deep feature vector is linearly transformed and activated by the Identity function through an output layer containing four neurons, resulting in a low-dimensional feature representation vector.

[0053] The low-dimensional feature representation vector has a dimension of 4.

[0054] Preferably, in step S42, for each dimension of the low-dimensional feature representation vector, the difference between each dimension value and the corresponding dimension value of the hypersphere center point is calculated, and the distance score is obtained by squaring and summing the differences corresponding to the four dimensions. The distance values ​​are then standardized to obtain a first-time anomaly detection score, the calculation expression of which includes:

[0055] ;

[0056] ;

[0057] in, This represents the distance score, and j represents the dimension index. This indicates the dimension value corresponding to the low-dimensional feature representation vector. This represents the dimension value corresponding to the center point of the hypersphere. This represents the anomaly score in a single detection, and e represents the natural constant.

[0058] Preferably, in step S43, a threshold judgment is performed on the anomaly score of a single detection to obtain a single detection result. A single detection result includes no threat and suspected security threat, specifically including:

[0059] When the anomaly score detected in a single instance is less than the preset initial screening value, the result of a single instance is that the log data corresponding to the log feature vector has no anomalies.

[0060] When the anomaly score detected in a single instance is greater than or equal to the preset initial screening score, the detection result indicates that the log data corresponding to the log feature vector has a suspected security threat.

[0061] A preliminary subset of abnormal logs is obtained by statistically analyzing the log data corresponding to a single detection result that indicates a suspected security threat.

[0062] Preferably, in step S51, each structured record in the preliminary anomaly log subset is treated as an event, and event traces are constructed to obtain the XES preliminary screening event log. The processing logic includes:

[0063] Based on triplet structured data, log records in the preliminary abnormal log subset are grouped to obtain behavioral sequences;

[0064] Extract the triplet structured data corresponding to the preliminary abnormal log subset, establish a unique case identifier (Case ID) based on the source IP, destination IP and transport layer protocol number in the identifier, and classify log records with the same unique case identifier into the same behavior sequence.

[0065] Within each behavior sequence, an activity name is established based on the action type in the corresponding behavior symbol and the service identifier in the feature symbol of each log record.

[0066] The timestamp order is obtained by sorting the log records in ascending order based on their corresponding timestamps.

[0067] The XES initial screening event log is generated based on the behavior sequence, activity name, and timestamp order, resulting in an extensible event stream format.

[0068] Step S52: The XES initial screening event log is processed using an inductive mining algorithm to obtain the workflow net to be detected. The workflow net to be detected includes places, transitions, and connection arcs. The processing logic includes:

[0069] The algorithm traverses the XES initial screening event log to obtain four basic relationships, and then splits the XES initial screening event log into mutually exclusive subsets based on the four basic relationships.

[0070] The four basic relations include sequence relation, selection relation, parallel relation, and cyclic relation;

[0071] The process tree is constructed from the bottom up based on mutually exclusive subsets. The internal nodes of the process tree are the operators corresponding to the four basic relations, and the leaf nodes of the process tree are the activity names.

[0072] By mapping leaf nodes to transitions and operators to places and connection arcs, a workflow network is obtained.

[0073] Preferably, step S61 involves performing a consistency check between the workflow net to be detected and a preset reference workflow net using an edit distance algorithm, including:

[0074] The workflow net to be tested is converted into a preset reference workflow net through editing operations.

[0075] Editing operations include transition operations, place operations, and connection arc operations;

[0076] The cost of a transition operation is 'a', which includes transition replacement, transition insertion, and transition deletion.

[0077] The cost of place operations is b, which includes place replacement, place insertion, and place deletion.

[0078] The cost of a connection arc operation is c, which includes connection arc insertion and connection arc deletion.

[0079] The minimum editing operation cost required to convert the workflow network to be detected into a preset reference workflow network is calculated using the edit distance algorithm. A secondary detection score is then calculated based on this minimum editing operation cost, and its calculation expression is as follows:

[0080] ;

[0081] ;

[0082] ;

[0083] Cost represents the cost of editing operations. Indicates the number of transition operations. Indicates the number of operations performed on the library. Indicates the number of connection arc operations. This indicates the score from the second test. Indicates the minimum cost of editing operations. Indicates the maximum cost of editing operations. This represents the total number of transitions in the workflow network to be detected. This represents the total number of transitions in the reference workflow network. This represents the total number of locations in the workflow network to be tested. This indicates the total number of reference workflow network locations. This indicates the total number of connection arcs in the workflow network to be tested. This represents the total number of connection arcs in the reference workflow net, and max(·) indicates taking the maximum value;

[0084] Step S62: Based on the secondary detection score, a secondary judgment is made to obtain the security detection result. The security detection result includes no threat and the presence of a security threat. The processing logic includes:

[0085] When the score of the secondary detection is greater than or equal to the preset judgment threshold, the security detection result is no threat;

[0086] When the score of the second detection is less than the preset judgment threshold, the security detection result is that there is a security threat.

[0087] Based on the security detection results, a unique case identifier is determined for the workflow network under test that has a security threat, and the structured log data and the corresponding raw log record data covered by the unique case identifier are output.

[0088] On the other hand, the present invention provides a firewall log security detection system, including: a data acquisition module, a preprocessing module, a feature extraction module, a primary detection module, a workflow network generation module, a secondary detection module, and a storage and display module;

[0089] The data acquisition module is used to collect raw log data from the firewall under test;

[0090] The preprocessing module is used to preprocess the raw log data to obtain triplet structured data;

[0091] The feature extraction module is used to perform numerical transformation and vector concatenation on triplet structured data to obtain log feature vectors;

[0092] The primary detection module is used to perform a primary detection on the log feature vectors using deep support vector data description to obtain a primary detection result and generate a preliminary subset of abnormal logs.

[0093] The workflow network generation module is used to construct event traces from a subset of preliminary abnormal logs to obtain XES initial screening event logs, and to generate a workflow network to be detected through an inductive mining algorithm.

[0094] The secondary detection module calculates the secondary detection score by performing a consistency check using the edit distance algorithm, and then performs a secondary judgment to obtain the security detection result.

[0095] The storage and display module stores all output data from the data acquisition module, preprocessing module, feature extraction module, primary detection module, workflow network generation module, and secondary detection module, and provides a visual interface to the user.

[0096] The beneficial effects of this invention are as follows: The final output of this application is not a single log data entry, but rather a sequence of behaviors that flag security threats, outputting all relevant log data. This makes isolated anomalies less likely to be misjudged, effectively reducing the false alarm rate. The secondary filtering mechanism of this application can efficiently process large-scale log data, ensuring the efficiency of editing cost distance calculation. Simultaneously, the construction of event traces and workflow networks transforms a large number of discrete preliminary anomaly log records into intuitive and interpretable graphical behavioral patterns, revealing the relationships between events. This allows security analysts to clearly see how anomalies occur and evolve, improving the interpretability of anomalies and enhancing the log security analysis and response process. Attached Figure Description

[0097] Figure 1 This is a basic flowchart illustrating a firewall log security detection method according to an embodiment of the present invention.

[0098] Figure 2 This is a schematic diagram of the process for obtaining triplet structured data according to an embodiment of the present invention;

[0099] Figure 3 This is a schematic diagram of the process of obtaining the workflow network to be detected, provided as an embodiment of the present invention.

[0100] Figure 4 A schematic diagram of the loss function for describing deep support vector data provided in one embodiment of the present invention.

[0101] Figure 5 This is a schematic diagram of the basic framework of a firewall log security detection system provided in one embodiment of the present invention. Detailed Implementation

[0102] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0103] Example 1, refer to Figure 1-4 As an embodiment of the present invention, a firewall log security detection method is provided, comprising:

[0104] Step S1: Collect raw log data from the firewall to be detected;

[0105] Step S2: Preprocess the original log data to obtain triplet structured data;

[0106] Step S3: Perform numerical transformation and vector concatenation on the triplet structured data to obtain the log feature vector;

[0107] Step S4: Perform a detection on the log feature vector using deep support vector data description to obtain a detection result and generate a preliminary subset of abnormal logs;

[0108] Step S5: Based on the preliminary abnormal log subset, construct event traces to obtain the XES preliminary screening event log, and generate the workflow network to be detected through inductive mining algorithm;

[0109] Step S6: The consistency check is performed using the edit distance algorithm to calculate the secondary detection score, and a secondary judgment is made to obtain the security detection result.

[0110] The core of the firewall log security detection method proposed in this invention lies in constructing a multi-stage, layered detection process. It first performs structured preprocessing and numerical transformation on the raw log data, then uses deep support vector data description for preliminary anomaly screening, generating a preliminary subset of anomalous logs focusing on potential threats. Furthermore, this application establishes a workflow network reflecting log behavior patterns, using an edit distance algorithm for consistency checks and secondary judgments to obtain security detection results. This method can accurately identify hidden security threats and effectively capture complex, combined, and sequential anomalous behaviors, rather than focusing solely on the anomalies of a single log record. Through two phased screening stages, this invention balances computational efficiency and accuracy, effectively handling large amounts of log data. Simultaneously, the introduction of a workflow network to abstract and compare the behavioral patterns reflected in the log data significantly improves the accuracy of detection and the interpretability of results, enabling security analysts to intuitively understand the process and root causes of anomalous behavior. It also reduces the false positive rate, providing a reliable and efficient solution for firewall log security detection.

[0111] In this embodiment, step S1 involves collecting raw log data from the firewall under test based on the Syslog protocol, including:

[0112] Listen on the firewall's UDP port 514 or TCP port 6514 using the Syslog protocol to receive key-value log data from the Syslog message stream conforming to the RFC5424 standard.

[0113] Log data in key-value pair format includes timestamps, action types, session statistics, source IP, destination IP, source port, destination port, transport layer protocol number, service identifier, security response type, client type, policy ID, and geographic information;

[0114] Action types include allow, deny, client reset, and timeout;

[0115] Session statistics include the number of bytes sent, the number of bytes received, and the session duration;

[0116] Security response types include logging, alarm generation, and connection reset;

[0117] Client types include browsers, web crawlers, and API tools.

[0118] In this application, log data collection is performed. By listening to and receiving standard Syslog message streams, the integrity and timeliness of the data source are ensured, providing a comprehensive information foundation for all subsequent analysis steps and facilitating the capture of real-time abnormal behavior records.

[0119] In this embodiment, step S21 involves parsing each record in the original log data using regular expressions to obtain a key-value pair dataset, including:

[0120] For numeric string values ​​and unquoted string values, the first capture instruction is executed to obtain the corresponding key-value pairs. The expression for the first capture instruction is:

[0121] ([^\\s]*)=([^"\\s]*)(\\s|$);

[0122] For string values ​​containing double quotes, the second capture instruction is executed to obtain the corresponding key-value pair. The expression for the second capture instruction is:

[0123] ([^\\s]*)="([^"]*)"(\\s|$);

[0124] Extract the timestamp field from the log records and convert it to the standardized ISO 8601 format;

[0125] Step S22: Based on the key-value pair dataset, fields are extracted and combined to obtain the triplet structured data corresponding to each log record. The expression for the triplet structured data is as follows:

[0126] [Identifier, Behavior Characteristic, Feature Characteristic];

[0127] The identifier includes the source IP, destination IP, source port, destination port, and transport layer protocol number;

[0128] The behavior identifier includes the action type, policy ID, and security response type;

[0129] The identifiers include session statistics, service identifier, client type, and geographic information.

[0130] This application efficiently and automatically transforms massive amounts of unstructured log data into unified and standardized triple-structured data, significantly reducing the complexity of data processing. Through predefined regular expressions, it can accurately parse key-value pairs in logs and construct standardized triple structures. This is particularly beneficial for subsequent accurate feature extraction and behavior sequence construction, avoiding the inefficiency and errors of manual parsing and ensuring data format consistency. Establishing standardized, semantic triple structures facilitates the efficient transformation of data into directly processable numerical feature vectors in subsequent steps and provides a natural semantic foundation for constructing behavioral patterns in step S5, ensuring data quality and efficiency throughout the entire analysis process.

[0131] In this embodiment, step S31 involves performing a first-order numerical conversion on the identifier, and the processing logic is as follows:

[0132] The source IP and destination IP in the identifier are encoded using IPv4 address to integer;

[0133] Perform outlier checks on the source and destination ports. If the integer value corresponding to the port does not exceed the range of 1 to 65535, directly retain the original integer value.

[0134] When the integer value corresponding to a port exceeds the range of 1 to 65535, the port will be uniformly mapped to 65536.

[0135] The transport layer protocol number is encoded using the IANA standard protocol number encoding, and the unknown protocol number is mapped to the value 0;

[0136] Step S32, perform a second numerical conversion on the behavior symbol, the processing logic includes:

[0137] The original integer value is directly retained for the policy ID in the behavior specification;

[0138] The action type and security response type are mapped and assigned values ​​using a preset first tag code. The mapping relationship is as follows:

[0139] Map allowed actions to 000, denied actions to 001, client reset actions to 002, and timeout actions to 003 in the action type;

[0140] Map the logging type in the security response category to 100, the alarm generation type to 101, and the connection reset type to 102.

[0141] This application transforms heterogeneous structured log information into numerical feature vectors that can be directly processed by machine learning models, eliminating data type differences. The standardization and numerical processing employed in this application facilitates input and efficient computation in subsequent steps, ensures the comparability of features across different dimensions, enhances the model's ability to capture potential anomalies, and provides a better data foundation for automated analysis.

[0142] Step S33, the third-order numerical transformation of the feature symbol, the processing logic is as follows:

[0143] Convert ISO 8601 format timestamps from string format to Unix format timestamps from integer format;

[0144] Z-score normalization is applied to the number of bytes sent, bytes received, and session duration in the session statistics.

[0145] The service identifier and client type are mapped and assigned using a preset second tag encoding. The mapping relationship is as follows:

[0146] Map HTTPS to 300, HTTP to 301, SSH to 302, FTP to 303, and other service identifiers to 304 in the service identifiers;

[0147] Map browsers to 400, web crawlers to 401, API tools to 402, and other client types to 403.

[0148] Geographic information is encoded and mapped according to the alpha-2 format of the ISO 3166-1 standard;

[0149] Step S34: The triplet structured data that has undergone numerical transformation is concatenated into vectors according to its original arrangement order to obtain a one-dimensional numerical log feature vector.

[0150] The transport layer protocol number uses the IANA standard protocol number, including TCP=6, UDP=17, ICMP=1, and unknown protocol numbers are mapped to 0.

[0151] In this embodiment, step S41 involves receiving the log feature vector through an input layer containing 15 neurons, and then sending each element of the log feature vector into a corresponding neuron in the input layer.

[0152] The log feature vector is processed by a first linear transformation and a ReLU function nonlinear activation to obtain the output vector of the first hidden layer. The first hidden layer contains 32 neurons.

[0153] The output vector of the first hidden layer is used as the input data of the second hidden layer. The second linear transformation and ReLU nonlinear activation are performed to complete forward propagation and obtain the deep feature vector. The second hidden layer contains 16 neurons.

[0154] The deep feature vector is linearly transformed and activated by the Identity function through an output layer containing four neurons, resulting in a low-dimensional feature representation vector.

[0155] The low-dimensional feature representation vector has a dimension of 4.

[0156] Deep SVDD is a mature anomaly detection model. In this application, the input layer is set to 15 neurons, which is beneficial for receiving the 15-dimensional log feature vector completely transformed from step S34, ensuring that all extracted raw log information is input without loss. The first hidden layer contains 32 neurons, which is greater than the input dimension. This is particularly beneficial for the network to learn and extract more complex and higher-order nonlinear feature combinations, providing sufficient capacity for subsequent feature abstraction. The second hidden layer contains 16 neurons, which is between the number of the first hidden layer and the output layer. This is particularly beneficial for further refining and compressing the features extracted by the first layer, gradually removing redundant information, and preparing for the construction of a compact latent representation. Finally, the output layer contains 4 neurons, generating a 4-dimensional low-dimensional feature representation vector, completing dimensionality compression. This is beneficial for tightly mapping and clustering normal log data into a low-dimensional space, thereby amplifying the distance difference between abnormal and normal data, adapting to the hypersphere calculation mode in Deep SVDD, and making anomalies more significant and easier to detect in subsequent distance calculations. The hierarchical design and neuron count configuration of the network together achieve a non-linear abstraction of the original log features, providing a highly discriminative representation for efficient anomaly identification using Deep Support Vector Data Description (Deep SVDD).

[0157] In this embodiment, step S42 involves calculating the difference between each dimension value and the dimension value corresponding to the center point of the hypersphere for each dimension of the low-dimensional feature representation vector, and then squaring and summing the differences for the four dimensions to obtain a distance score. The distance values ​​are then standardized to obtain a single anomaly detection score, the calculation of which includes:

[0158] ;

[0159] ;

[0160] in, This represents the distance score, and j represents the dimension index. This indicates the dimension value corresponding to the low-dimensional feature representation vector. This represents the dimension value corresponding to the center point of the hypersphere. This represents the anomaly score in a single detection, and e represents the natural constant.

[0161] In this application, the mature Deep Support Vector Data Description (Deep SVDD) is used to filter the log data. In this embodiment, the Deep Support Vector Data Description (Deep SVDD) is trained based on 60 days of historical normal firewall log data, and the compact representation in the low-dimensional feature space is used to determine the center point and radius of the hypersphere.

[0162] In this embodiment, the Deep Support Vector Data Description (Deep SVDD) model is trained on 60 days of historical normal firewall log data. The training process includes: a hidden layer network converting the input data into a low-dimensional representation, randomly initializing the connection parameters of the hidden layer, and randomly setting a center point in the low-dimensional space. In the iterative optimization phase, in each iteration, the model processes samples extracted from the historical normal log data. First, the hidden layer network transforms the samples to obtain corresponding representations in the low-dimensional space. Then, the deviation between these low-dimensional representations and the preset center point is calculated. A loss function is constructed to minimize the distance of all normal data points in the low-dimensional space to the center point, while minimizing the area containing these normal points. Gradient descent is used to iteratively adjust the connection parameters of the hidden layer network and the position of the center point until a preset maximum of 30 training iterations is reached. Based on the connection parameters minimized by the loss function, the hidden layer network is determined, and the hypersphere center point is also determined, completing the training.

[0163] This approach fully leverages the low training cost and compact representation of normal data by deep support vector data description to efficiently identify potential anomalous data that deviates from normal patterns. This reduces the amount of data requiring complex calculations such as edit distance in subsequent stages, improving the overall efficiency of the detection system, effectively reducing subsequent computational resource consumption, saving costs, and enhancing the adaptability of the solution.

[0164] Step S43: A threshold judgment is applied to the anomaly score detected in a single detection to obtain a single detection result. A single detection result includes either no threat or suspected security threat, specifically including:

[0165] When the anomaly score detected in a single instance is less than the preset initial screening value, the result of a single instance is that the log data corresponding to the log feature vector has no anomalies.

[0166] When the anomaly score detected in a single instance is greater than or equal to the preset initial screening score, the detection result indicates that the log data corresponding to the log feature vector has a suspected security threat.

[0167] A preliminary subset of abnormal logs is obtained by statistically analyzing the log data corresponding to a single detection result that indicates a suspected security threat.

[0168] The anomaly score of the current log record is mapped to the interval [0, 1), which quantitatively reflects the degree of deviation between the feature representation of the log record and the center of normal behavior. The closer the score is to 1, the greater the deviation, and the higher the probability that the log is a security threat. This is helpful for intuitively showing the results of a screening.

[0169] Step S51: Treat each structured record in the preliminary anomaly log subset as an event, and construct event traces to obtain the XES preliminary event log. The processing logic includes:

[0170] Based on triplet structured data, log records in the preliminary abnormal log subset are grouped to obtain behavioral sequences;

[0171] Extract the triplet structured data corresponding to the preliminary abnormal log subset, establish a unique case identifier (Case ID) based on the source IP, destination IP and transport layer protocol number in the identifier, and classify log records with the same unique case identifier into the same behavior sequence.

[0172] Within each behavior sequence, an activity name is established based on the action type in the corresponding behavior symbol and the service identifier in the feature symbol of each log record.

[0173] The timestamp order is obtained by sorting the log records in ascending order based on their corresponding timestamps.

[0174] The XES initial screening event log is generated based on the behavior sequence, activity name, and timestamp order, resulting in an extensible event stream format.

[0175] Step S52: The XES initial screening event log is processed using an inductive mining algorithm to obtain the workflow net to be detected. The workflow net to be detected includes places, transitions, and connection arcs. The processing logic includes:

[0176] The algorithm traverses the XES initial screening event log to obtain four basic relationships, and then splits the XES initial screening event log into mutually exclusive subsets based on the four basic relationships.

[0177] The four basic relations include sequence relation, selection relation, parallel relation, and cyclic relation;

[0178] The process tree is constructed from the bottom up based on mutually exclusive subsets. The internal nodes of the process tree are the operators corresponding to the four basic relations, and the leaf nodes of the process tree are the activity names.

[0179] By mapping leaf nodes to transitions and operators to places and connection arcs, a workflow network is obtained.

[0180] This application introduces a workflow network to capture the attributes of log events, representing the dynamic relationships between events such as sequence, parallelism, and selection, avoiding the limitations of traditional methods that only focus on individual log records or isolated features. This improves the interpretability of security detection, enabling security analysts to intuitively understand the development of abnormal behavior, rather than simply obtaining an abstract anomaly score or warning. This is beneficial for analyzing the root causes of security problems and developing response strategies, while also demonstrating high adaptability to detecting unknown attacks. Furthermore, unlike the aforementioned single-stage screening, the workflow network does not use all features in the triplet structured data, but focuses only on a subset of features, enabling the abstraction and comparison of behavioral processes and avoiding computational overload caused by considering all features.

[0181] In this embodiment, step S61, which involves performing a consistency check between the workflow network to be detected and a preset reference workflow network using an edit distance algorithm, includes:

[0182] The workflow net to be tested is converted into a preset reference workflow net through editing operations.

[0183] Editing operations include transition operations, place operations, and connection arc operations;

[0184] The cost of a transition operation is 'a', which includes transition replacement, transition insertion, and transition deletion.

[0185] The cost of place operations is b, which includes place replacement, place insertion, and place deletion.

[0186] The cost of a connection arc operation is c, which includes connection arc insertion and connection arc deletion.

[0187] The minimum editing operation cost required to convert the workflow network to be detected into a preset reference workflow network is calculated using the edit distance algorithm. A secondary detection score is then calculated based on this minimum editing operation cost, and its calculation expression is as follows:

[0188] ;

[0189] ;

[0190] ;

[0191] Cost represents the cost of editing operations. Indicates the number of transition operations. Indicates the number of operations performed on the library. Indicates the number of connection arc operations. This indicates the score from the second test. Indicates the minimum cost of editing operations. Indicates the maximum cost of editing operations. This represents the total number of transitions in the workflow network to be detected. This represents the total number of transitions in the reference workflow network. This represents the total number of locations in the workflow network to be tested. This indicates the total number of reference workflow network locations. This indicates the total number of connection arcs in the workflow network to be tested. This represents the total number of connection arcs in the reference workflow net, and max(·) indicates taking the maximum value;

[0192] The preset reference workflow network is identical to the training data of the aforementioned Deep Support Vector Data Description (Deep SVDD) model, consisting of 60 days of historical normal firewall log data. By applying the event trace construction and inductive mining algorithm in step S5 of this invention to the 60 days of historical normal firewall log data, the normal behavior pattern of the system is abstracted.

[0193] The cost settings for transition operations, place operations, and connection arc operations are designed to better reflect the impact of different editing operations on the structural and semantic differences between the two workflow networks. In this embodiment, value 'a' is set to 3, value 'b' to 1, and value 'c' to 2, fully reflecting the principle that transition operations have the highest cost, followed by connection arc operations, and place operations have the lowest cost. This helps to more effectively highlight more threatening behavioral pattern changes when calculating edit distance, enabling the final secondary detection score to more accurately reflect the actual threat level.

[0194] Specifically, an edit distance algorithm is used to perform a consistency check between the workflow network to be inspected, which may have security vulnerabilities, and a preset reference workflow network, and a secondary detection score is calculated. This can quantify the structural and semantic differences between the two workflow networks, especially allowing for a comparison at the essential level of behavioral processes. Step S62 involves a secondary judgment based on the secondary detection score to obtain a security detection result, which includes no threat and the presence of a security threat. The processing logic includes:

[0195] When the score of the secondary detection is greater than or equal to the preset judgment threshold, the security detection result is no threat;

[0196] When the score of the secondary detection is less than the preset judgment threshold, the security detection result is that there is a security threat.

[0197] Based on the security detection results, a unique case identifier is determined for the workflow network under test that has a security threat, and the structured log data and the corresponding raw log record data covered by the unique case identifier are output.

[0198] Unlike classification methods that rely solely on numerical features, this application uses edit distance to delve into the essential level of behavioral processes for comparison. This accurately identifies hidden anomalies such as disordered activity sequences, additions or deletions of key activities, or alterations to process logic, rather than simply deviations in feature values. This effectively filters out false alarms that may occur in the first stage, improving the accuracy and reliability of anomaly detection.

[0199] Traditional methods and the initial screening in this application focus only on the numerical characteristics of individual log entries. This application uses the initial screening result as an intermediate measure, determining the final security detection result through a secondary detection. By employing edit distance to delve into the essential level of behavioral processes, it can accurately identify anomalies not recorded in individual log entries, such as disordered activity sequences and altered process logic. This effectively filters out false positives that may arise during the initial screening, thereby improving the accuracy and reliability of anomaly detection. Finally, when a security threat is detected, this application obtains a unique case identifier corresponding to the anomaly workflow network, outputting all structured log data and raw log records it encompasses. This is particularly beneficial for security analysts to quickly understand the complete context of the anomaly and accelerates subsequent threat tracing, response, and forensics.

[0200] Example 2, refer to Figure 5 According to one embodiment of the present invention, a firewall log security detection system is provided, including: a data acquisition module, a preprocessing module, a feature extraction module, a primary detection module, a workflow network generation module, a secondary detection module, and a storage and display module;

[0201] The data acquisition module is used to collect raw log data from the firewall under test;

[0202] The preprocessing module is used to preprocess the raw log data to obtain triplet structured data;

[0203] The feature extraction module is used to perform numerical transformation and vector concatenation on triplet structured data to obtain log feature vectors;

[0204] The primary detection module is used to perform a primary detection on the log feature vectors using deep support vector data description to obtain a primary detection result and generate a preliminary subset of abnormal logs.

[0205] The workflow network generation module is used to construct event traces from a subset of preliminary abnormal logs to obtain XES initial screening event logs, and to generate a workflow network to be detected through an inductive mining algorithm.

[0206] The secondary detection module calculates the secondary detection score by performing a consistency check using the edit distance algorithm, and then performs a secondary judgment to obtain the security detection result.

[0207] The storage and display module stores all output data from the data acquisition module, preprocessing module, feature extraction module, primary detection module, workflow network generation module, and secondary detection module, and provides a visual interface to the user.

[0208] Those skilled in the art will understand that embodiments of the present invention can provide methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0209] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A firewall log security detection method, characterized by, The method comprises the following steps: Step S1, collecting original log record data of a to-be-detected firewall; Step S2, preprocessing the original log record data to obtain triadic structure data; Step S3, performing numerical conversion and vector splicing on the triadic structure data to obtain a log feature vector; Step S4, performing primary detection on the log feature vector by deep support vector data description to obtain a primary detection result and generate a preliminary abnormal log subset; Step S5, constructing an event trace based on the preliminary abnormal log subset to obtain an XES preliminary screening event log, and generating a to-be-detected workflow net by an inductive mining algorithm; Step S6, performing consistency check calculation by an edit distance algorithm to obtain a secondary detection score, and performing secondary judgment to obtain a security detection result; The expression of the triadic structure data is: [identifier, behavior symbol, feature symbol]; The identifier comprises a source IP, a destination IP, a source port, a destination port and a transmission layer protocol number; The behavior symbol comprises an action type, a policy ID and a security response type; The feature symbol comprises session statistics, a service identifier, a client type and geographic information; Step S5 comprises steps S51 and S52, and the details are as follows: Step S51, regarding each structured record in the preliminary abnormal log subset as an event (Event), constructing an event trace to obtain an XES preliminary screening event log, and the processing logic comprises: Grouping log records in the preliminary abnormal log subset based on triadic structure data to obtain a behavior sequence; Extracting triadic structure data corresponding to the preliminary abnormal log subset, establishing a unique case identifier (Case ID) based on the source IP, the destination IP and the transmission layer protocol number in the identifier, and classifying log records with the same unique case identifier into the same behavior sequence; In each behavior sequence, establishing an activity name (Activity Name) based on the action type in the behavior symbol and the service identifier in the feature symbol of each log record; Arranging time stamps in ascending order to obtain a time stamp sequence; Based on the behavior sequence, the activity name and the time stamp sequence, an XES preliminary screening event log in an extensible event stream format is established; Step S52, processing the XES preliminary screening event log by an inductive mining algorithm (Inductive Miner) to obtain a to-be-detected workflow net (Workflow Net), wherein the to-be-detected workflow net comprises a library, a transition and a connection arc, and the processing logic of the inductive mining algorithm comprises: The algorithm traverses the XES preliminary screening event log to obtain four basic relationships, and the XES preliminary screening event log is split into mutually exclusive subsets based on the four basic relationships; The four basic relationships comprise a sequential relationship, a selection relationship, a parallel relationship and a cycle relationship; A process tree is constructed from bottom to top based on the mutually exclusive subsets, wherein the internal nodes of the process tree are operators corresponding to the four basic relationships, and the leaf nodes of the process tree are activity names; The leaf nodes are mapped to transitions, and the operators are mapped to libraries and connection arcs, to obtain a workflow net.

2. The firewall log security detection method of claim 1, wherein: Step S1 comprises: collecting the original log record data based on the Syslog protocol, and the specific steps are as follows: Through the Syslog protocol, the UDP 514 port or the TCP 6514 port of the firewall is monitored, and the Syslog message stream of the RFC 5424 standard is received to obtain log data in the form of key-value pairs; The log data in the form of key-value pairs includes a timestamp, an action type, session statistics, a source IP, a destination IP, a source port, a destination port, a transport layer protocol number, a service identifier, a security response type, a client type, a policy ID, and geographic information; The action type includes allowing, rejecting, client resetting, and timeout; The session statistics include the number of sent bytes, the number of received bytes, and session duration; The security response type includes logging, alarm generation, and connection resetting; The client type includes a browser, a crawler tool, and an API tool.

3. The firewall log security detection method of claim 1, wherein: Step S2 comprises steps S21 and S22, and the original log record data is preprocessed to obtain structured data in a triple form, and the specific steps are as follows: Step S21, regular expression analysis is performed on each record in the original log record data to obtain a key-value pair data set, including: For numerical string values and unquoted string values, a first capture instruction is executed to obtain the corresponding key-value pair, and the expression of the first capture instruction is as follows: ([^\\s]*)=([^"\\s]*)(\\s|$); For string values containing double quotes, a second capture instruction is executed to obtain the corresponding key-value pair, and the expression of the second capture instruction is as follows: ([^\\s]*)="([^"]*)"(\\s|$); The timestamp field in the log record is extracted and uniformly converted into a standardized ISO 8601 format; Step S22, field extraction and combination are performed based on the key-value pair data set to obtain structured data in a triple form corresponding to each log record.

4. The firewall log security detection method of claim 1, wherein: Step S3 comprises steps S31, S32, S33, and S34, and the structured data in a triple form is subjected to numerical conversion and vector splicing to obtain a log feature vector, and the specific steps are as follows: Step S31, first numerical conversion is performed on the identifier, and the processing logic is as follows: IPv4 address-to-integer encoding is performed on the source IP and the destination IP in the identifier; Abnormal value judgment is performed on the source port and the destination port, and when the integer value corresponding to the port does not exceed the range of 1 to 65535, the original integer value is directly retained; When the integer value corresponding to the port exceeds the range of 1 to 65535, the port is uniformly mapped to 65536; IANA standard protocol number coding is performed on the transport layer protocol number, and unknown protocol numbers are mapped to the value 0; Step S32, second numerical conversion is performed on the behavior symbol, and the processing logic includes: The policy ID in the behavior symbol is directly retained in the original integer value; The action type and the security response type are mapped and assigned by a preset first label code; Step S33, third numerical conversion is performed on the feature symbol, and the processing logic is as follows: The ISO 8601 format timestamp in the form of a string is converted into a Unix timestamp in the form of an integer; The number of transmitted bytes, the number of received bytes and the session duration in the session statistics are Z-score standardized; The service identifier and the client type are mapped and assigned values by using a preset second label code; The geographic information is encoded and mapped according to the alpha-2 format of the ISO 3166-1 standard; Step S34: The triadic structured data after the numerical conversion is spliced in the original arrangement order to obtain a one-dimensional numerical log feature vector.

5. The firewall log security detection method of claim 1, wherein: Step S4 includes steps S41, S42 and S43, and specifically as follows: Step S41: A low-dimensional feature representation vector is obtained by processing the received log feature vector through deep support vector data description (Deep SVDD), and the processing logic includes: The log feature vector is received by an input layer containing 15 neurons, and each element in the log feature vector is sent to a corresponding neuron in the input layer; A first hidden layer output vector is obtained by performing a first linear transformation and a ReLU function nonlinear activation process on the log feature vector, and the first hidden layer contains 32 neurons; The first hidden layer output vector is taken as input data of a second hidden layer, and a second linear transformation and a ReLU function nonlinear activation process are performed to complete forward propagation and obtain a deep feature vector, and the second hidden layer contains 16 neurons; The deep feature vector is linearly transformed and Identity function linearly activated by an output layer containing 4 neurons to output a low-dimensional feature representation vector. The low-dimensional feature representation vector has a dimension of 4.

6. The firewall log security detection method of claim 5, wherein: Step S42: For each dimension of the low-dimensional feature representation vector, the difference between the dimension value and the corresponding dimension value of the hypersphere center point is calculated, and the square sum of the differences of the four dimensions is calculated to obtain a distance score, and the distance value is standardized to obtain a first detection anomaly score, and the calculation expression includes: ； ； wherein, denotes a distance score, j denotes a dimension index, denotes a dimension value corresponding to the low-dimensional feature representation vector, denotes a dimension value corresponding to the hypersphere center point, denotes a one-time detection anomaly score, e denotes a natural constant.

7. The firewall log security detection method of claim 6, wherein: Step S43: The first detection anomaly score is subjected to threshold value judgment to obtain a first detection result, and the first detection result includes no threat and suspected security threat, and specifically includes: When the first detection anomaly score is less than a preset preliminary screening value, the first detection result is that the log data corresponding to the log feature vector has no anomaly; When the first detection anomaly score is greater than or equal to the preset preliminary screening value, the first detection result is that the log data corresponding to the log feature vector has a suspected security threat; The log data corresponding to the suspected security threat in the first detection result is counted to obtain a preliminary abnormal log subset.

8. The firewall log security detection method of claim 1, wherein: Step S6 includes steps S61 and S62, and specifically as follows: Step S61: The consistency of the to-be-detected workflow network and the preset reference workflow network is checked by using an edit distance algorithm, including: The to-be-detected workflow network is converted into the preset reference workflow network by an edit operation, The edit operation includes a transition operation, a library operation and a connection arc operation; The cost of the transition operation is a, including transition replacement, transition insertion and transition deletion; The cost of the library operation is b, including library replacement, library insertion and library deletion; The cost of the connection arc operation is c, including connection arc insertion and connection arc deletion; The minimum editing operation cost required for converting the to-be-detected workflow net into the preset reference workflow net is calculated through an edit distance algorithm, and a secondary detection score is calculated based on the minimum editing operation cost, and the calculation expression is: ； ； ； Cost represents the edit operation cost, represents the number of transition operations, represents the number of pool operations, represents the number of connection arc operations, represents the secondary detection score, represents the minimum edit operation cost, represents the maximum edit operation cost, represents the total number of transitions of the workflow net to be detected, represents the total number of transitions of the reference workflow net, represents the total number of pools of the workflow net to be detected, represents the total number of pools of the reference workflow net, represents the total number of connection arcs of the workflow net to be detected, represents the total number of connection arcs of the reference workflow net, and max(·) represents taking the maximum value. In step S62, a secondary judgment is performed based on the secondary detection score to obtain a security detection result, and the security detection result includes no threat and existence of security threat, and the processing logic includes: When the secondary detection score is greater than or equal to a preset judgment threshold, the security detection result is no threat; When the secondary detection score is less than the preset judgment threshold, the security detection result is existence of security threat; When the security detection result is existence of security threat, a unique case identifier corresponding to the to-be-detected workflow net with security threat is determined based on the security detection result, and the structured log data covered by the unique case identifier and the corresponding original log record data are output.

9. A firewall log security detection system for implementing the method of any one of claims 1-8, characterized by It includes: A data acquisition module, a preprocessing module, a feature extraction module, a primary detection module, a workflow net generation module, a secondary detection module and a storage display module; The data acquisition module is used to acquire original log record data from the to-be-detected firewall; The preprocessing module is used to preprocess the original log record data to obtain triple structured data; The feature extraction module is used to convert the triple structured data into numerical values and splice vectors to obtain a log feature vector; The primary detection module is used to perform primary detection on the log feature vector through deep support vector data description to obtain a primary detection result and generate a preliminary abnormal log subset; The workflow net generation module is used to construct event traces from the preliminary abnormal log subset to obtain an XES preliminary screening event log, and generate a to-be-detected workflow net through an inductive mining algorithm; The secondary detection module calculates a secondary detection score through consistency checking by an edit distance algorithm, and performs secondary judgment to obtain a security detection result; The storage display module is used to store all output data of the data acquisition module, the preprocessing module, the feature extraction module, the primary detection module, the workflow net generation module and the secondary detection module, and provides a visual interface to the user.

Images