A firewall-based network security detection method and system

Abstract

This invention discloses a firewall-based network security detection method and system, belonging to the field of network security. The method includes: discovering abnormal traffic patterns and outliers; identifying potential malicious traffic clusters based on clustering results; extracting representative samples from suspected malicious traffic clusters; matching the sample traffic with a known malicious traffic feature library via a threat intelligence interface; if the matching degree exceeds a preset threshold, it is determined to be malicious traffic of a known type; deploying a trained malicious traffic classification model to the firewall's online detection module to predict and classify real-time network traffic; if determined to be malicious traffic, triggering the firewall's blocking and isolation policies to intercept the malicious traffic in real time.

Description

Technical Field

[0001] This invention belongs to the field of network security, and in particular relates to a network security detection method and system based on a firewall. Background Technology

[0002] Deploying firewalls in enterprise networks presents a significant technical challenge: effectively identifying and blocking malicious traffic. While firewalls can filter traffic based on preset rules, relying solely on rule matching is insufficient for accurately identifying unknown or variant malicious traffic. Furthermore, the constantly evolving characteristics of malicious traffic and the lag in rule updates and maintenance also impact firewall detection effectiveness. In addition, the massive volume and complexity of network traffic pose challenges to the firewall's retrieval performance and efficiency.

[0003] How to quickly retrieve and identify malicious traffic, and promptly block and isolate it without affecting normal business traffic, has become a pressing technical challenge. How to achieve real-time integration and synchronous updates between firewalls and external security threat intelligence databases, how to build an efficient malicious traffic signature database and retrieval mechanism within the firewall, and how to balance the comprehensiveness of the retrieval with performance overhead are all technical problems that need to be overcome in practical applications. This requires innovation and optimization in firewall architecture, retrieval algorithms, and data synchronization, and the comprehensive application of multiple detection technologies to achieve accurate identification and effective defense against malicious traffic, thus ensuring network security. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a firewall-based network security detection method, comprising:

[0005] Obtain network traffic data packets, parse the key fields of the data packets, and obtain structured traffic metadata;

[0006] The structured traffic metadata is subjected to feature engineering to generate feature vectors that reflect traffic behavior patterns.

[0007] Cluster analysis is performed on the feature vectors reflecting traffic behavior patterns based on unsupervised machine learning algorithms, and suspected malicious traffic clusters are identified based on the clustering results.

[0008] Extract representative samples from the suspected malicious traffic clusters, and perform similarity matching between the sample traffic and the known malicious traffic feature library through the threat intelligence interface. If the matching degree exceeds a preset threshold, it is determined to be malicious traffic of a known type.

[0009] A malicious traffic classification model is obtained by training a support vector machine model and a decision tree using traffic samples annotated by experts.

[0010] The known types of malicious traffic are used as labels and fed back into the malicious traffic classification model for optimization. The optimized model is then used to detect network security issues.

[0011] Preferably, the process of obtaining structured traffic metadata includes:

[0012] Deep packet inspection technology is used to parse captured network data packets and extract key field information;

[0013] The key field information is processed to obtain the structured traffic metadata; wherein, if the protocol type is TCP, the network connection status and abnormal conditions are determined based on information such as source IP, destination IP and port number; if the protocol type is HTTP, the HTTP request and response data are parsed to extract the URL, request method and status code.

[0014] Preferably, the process of performing feature engineering on the structured traffic metadata to generate feature vectors reflecting traffic behavior patterns includes:

[0015] The structured traffic metadata is divided according to a preset time window size to obtain a time window;

[0016] The packet size distribution characteristics are obtained by analyzing the distribution of data packets within each time window based on the traffic data.

[0017] Calculate the interval between the arrival times of adjacent data packets within each time window, statistically analyze the distribution of these time intervals, and obtain the distribution characteristics of the packet arrival time intervals.

[0018] Determine the start and end times of each flow, calculate the duration of the flow, and statistically analyze the distribution of flow durations to obtain flow duration characteristics;

[0019] The packet size distribution feature, the packet arrival time interval distribution feature, and the flow duration feature are combined to obtain the feature vector reflecting the flow behavior pattern.

[0020] Preferably, the process of determining suspected malicious traffic clusters based on clustering results includes:

[0021] The K-means algorithm is used to cluster the feature vectors that reflect traffic behavior patterns, dividing the traffic into multiple clusters.

[0022] Calculate the distance from each traffic sample to the center of its cluster. If the distance exceeds a preset threshold, it is determined to be an anomaly.

[0023] The proportion of anomalies within each cluster is counted. If the proportion exceeds a preset threshold, the cluster is determined to be a malicious traffic cluster.

[0024] Train an autoencoder model to reduce the dimensionality and reconstruct the traffic features, calculate the reconstruction error, and if the reconstruction error of a traffic sample exceeds a preset threshold, it is judged as a suspected malicious traffic cluster.

[0025] Preferably, the process of matching the sample traffic with a known malicious traffic feature database includes:

[0026] Data on suspected malicious traffic clusters is obtained, the traffic clusters are analyzed, and the center point of each traffic cluster is used as a representative sample.

[0027] The extracted representative sample data is sent to the threat intelligence platform through the threat intelligence interface;

[0028] After receiving representative sample data, the threat intelligence platform calls a similarity matching algorithm to calculate the similarity between the sample and the data in the known malicious traffic feature database;

[0029] Based on the similarity value calculated by the similarity matching algorithm, determine whether it exceeds the preset similarity threshold;

[0030] If the similarity value exceeds the preset threshold, the traffic cluster containing the representative sample is identified as the corresponding type of known malicious traffic.

[0031] Preferably, the process of training the support vector machine model and decision tree using traffic samples annotated by experts to obtain a malicious traffic classification model includes:

[0032] Obtain malicious traffic sample data annotated by experts, extract features from the traffic sample data to obtain traffic feature vectors, construct a malicious traffic classification model using the support vector machine algorithm based on the traffic feature vectors, and obtain the support vector machine classifier through model training;

[0033] Based on the traffic feature vector, a malicious traffic classification model is constructed using the decision tree algorithm, and a decision tree classifier is obtained through model training;

[0034] The malicious traffic classification model is generated based on the support vector machine classifier and the decision tree classifier.

[0035] Preferably, the process of detecting network security based on the optimization model includes:

[0036] Obtain network traffic data, extract features from the traffic data and vectorize the data to obtain traffic feature vectors;

[0037] The traffic feature vector is input into the optimization model, and the malicious probability value of the traffic is obtained by model prediction.

[0038] Determine whether the probability value of malicious traffic exceeds a preset malicious threshold. If it exceeds the threshold, the traffic is determined to be malicious traffic; otherwise, it is determined to be normal traffic.

[0039] For traffic identified as malicious, the firewall's blocking policy is triggered to block the source and destination IP addresses of the malicious traffic in real time. At the same time, the malicious traffic is diverted to an isolation zone for in-depth analysis to extract key characteristics and attack methods.

[0040] Based on the malicious traffic characteristics obtained from the analysis and the known types of malicious traffic, the malicious traffic classification model is incrementally trained and optimized. The optimized malicious traffic classification model is then redeployed to the real-time detection module of the firewall for continuous iteration and updates, thereby achieving dynamic detection and real-time defense against malicious traffic.

[0041] On the other hand, the present invention also provides a firewall-based network security detection system, comprising:

[0042] The data acquisition module is used to acquire network traffic data packets, parse the key fields of the data packets, and obtain structured traffic metadata.

[0043] The processing module is used to perform feature engineering on the structured traffic metadata to generate feature vectors that reflect traffic behavior patterns.

[0044] The suspected malicious traffic cluster acquisition module is used to perform cluster analysis on the feature vectors reflecting traffic behavior patterns based on unsupervised machine learning algorithms, and determine suspected malicious traffic clusters based on the clustering results.

[0045] The determination module is used to extract representative samples of the suspected malicious traffic clusters, and perform similarity matching between the sample traffic and the known malicious traffic feature library through the threat intelligence interface. If the matching degree exceeds a preset threshold, it is determined to be malicious traffic of a known type.

[0046] The model building module is used to train the support vector machine model and decision tree using traffic samples annotated by experts to obtain a malicious traffic classification model.

[0047] The detection module is used to feed the known types of malicious traffic as labels into the malicious traffic classification model for optimization, and to detect network security based on the optimized model.

[0048] On the other hand, the present invention also provides an electronic device, including a memory, a processor, and a computing program stored in the memory and executable on the processor, wherein the processor implements the firewall-based network security detection method when executing the computer program.

[0049] On the other hand, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the firewall-based network security detection method.

[0050] Compared with the prior art, the present invention has the following advantages and technical effects:

[0051] This invention discloses a firewall-based network security detection method. This method captures and parses network packets in real time, extracts multi-dimensional traffic features, employs unsupervised clustering to discover anomaly patterns, and combines threat intelligence matching and supervised classification to identify malicious traffic. The invention deploys a trained classification model to the firewall's online detection module to achieve real-time interception of malicious traffic. By continuously collecting new malicious samples and dynamically updating the feature library and detection model, this invention can adaptively evolve, improving its ability to identify unknown and variant malicious traffic. This method integrates traffic capture, feature engineering, machine learning, and threat intelligence technologies, achieving intelligent and automated network security protection and effectively enhancing the firewall's ability to detect and defend against complex network attacks. Attached Figure Description

[0052] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0053] Figure 1 This is a schematic diagram of the method flow according to an embodiment of the present invention;

[0054] Figure 2 This is a schematic diagram of the system structure according to an embodiment of the present invention. Detailed Implementation

[0055] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0056] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0057] Example 1

[0058] like Figure 1-2 As shown, this embodiment provides a firewall-based network security detection method, including:

[0059] S101. Obtain network traffic data. Capture network packets passing through the firewall in real time through traffic mirroring or bypass packet capture, and parse key fields of the packets, such as source IP, destination IP, port number, protocol type, etc., to obtain structured traffic metadata.

[0060] Based on a pre-defined network topology, traffic mirroring or bypass packet capture devices are deployed at key nodes to acquire network packets passing through the firewall in real time. Deep packet inspection (DPI) technology is used to parse the captured network packets and extract key fields such as source IP, destination IP, port number, and protocol type. For the extracted key fields, feature engineering is used to convert the unstructured raw data into structured metadata. If the protocol type is TCP, the network connection status and anomalies are determined based on the source IP, destination IP, and port number. If the protocol type is HTTP, HTTP request and response data are further parsed to extract application layer information such as URL, request method, and status code. Machine learning algorithms such as decision trees and random forests are used to train an anomaly detection model based on network traffic metadata, enabling real-time early warning of network security threats. Based on the anomaly detection results, security policies are dynamically adjusted in conjunction with firewalls and other security devices to block malicious traffic and ensure network security.

[0061] Specifically, the network topology can be preset as a typical enterprise network, including the intranet, DMZ, and the internet. Key nodes such as the firewall's internal and external network interfaces, core switches, and front-end switches of DMZ servers are ideal locations for deploying traffic mirroring or bypass packet capture devices. For example, deploying a professional traffic mirroring device at the firewall's egress point can replicate traffic to an intrusion detection system (IDS) in real time. Deep packet inspection (DPI) technology can identify and analyze application layer protocols in network packets. For example, Wireshark is a commonly used DPI tool that can capture and parse network traffic using various protocols. Suppose a packet is captured with a source IP of 192.168.1.10, a destination IP of 8.8.8.8, port number 53, and UDP protocol. This indicates that the intranet host 192.168.1.10 is initiating a DNS query to the public DNS server 8.8.8.8. The extracted key field information needs to undergo feature engineering. For example, the source IP, destination IP, and port number can be directly used as numerical features. Protocol types can be converted into categorical features, such as 1 for TCP, 2 for UDP, and 3 for HTTP. Date and time can be converted into timestamp features. These features constitute structured metadata, facilitating subsequent processing by machine learning models. For the TCP protocol, connection status can be determined by analyzing TCP flags and sequence numbers. For example, the SYN flag indicates a connection is established, and the FIN flag indicates a connection is closed. If a large number of SYN requests occur in a short period without corresponding ACK responses, it may indicate a SYNFlood attack. Suppose a TCP connection with a source IP of 192.168.1.10, a destination IP of 192.168.1.20, and port number 80 is captured, and its status continuously switches between SYN_SENT and SYN_RECEIVED, which may indicate an anomaly in the connection. For the HTTP protocol, further analysis of application layer information is required. For example, extracting information such as the URL, request method (GET, POST, etc.), and User-Agent from the HTTP request. Suppose an HTTP request is captured with the URL ` / admin / login.php` and the request method is POST, which may indicate that someone is attempting to log in to the backend management interface. Machine learning algorithms can be used to detect network anomalies. For example, a decision tree algorithm can be used to build an anomaly detection model. The model's input is feature-engineered metadata, and its output is a prediction of whether the network is normal or abnormal. The model's training data can be historical network traffic data, containing known samples of normal and abnormal traffic. If the model detects an HTTP request whose destination IP is a known malicious IP address, it can determine that the request is abnormal. Based on the anomaly detection result, security devices such as firewalls can be activated to respond.For example, if a SYNFlood attack is detected, the firewall can be configured to dynamically limit the rate of SYN requests, or the attack source IP can be directly blocked. If an internal network host is detected accessing a known malicious website, the firewall can be configured to block that host from accessing the website and send an alert to the administrator. Doing so can effectively block malicious traffic and improve network security.

[0062] S102. Perform feature engineering on traffic metadata to extract multi-dimensional statistical features of traffic, such as packet size distribution, packet arrival time interval distribution, and flow duration. Through time window partitioning and aggregation operations, generate feature vectors that reflect traffic behavior patterns.

[0063] Network traffic metadata, including packet timestamps, packet sizes, source and destination IPs, and ports, is acquired and stored in a database. The traffic metadata is divided into a series of time windows, such as 1 second or 1 minute. For the traffic data within each time window, the distribution of packet sizes is statistically analyzed, including average packet size, maximum and minimum packet sizes, yielding packet size distribution characteristics. The arrival time intervals of adjacent packets within each time window are calculated, and the distribution of these time intervals, such as average interval and maximum and minimum intervals, is analyzed, yielding packet arrival time interval distribution characteristics. The start and end times of each flow are determined, and the flow duration is calculated, analyzing its distribution to obtain flow duration characteristics. These multi-dimensional statistical features are combined to form a high-dimensional feature vector reflecting traffic behavior patterns. Machine learning algorithms, such as clustering and anomaly detection, are used to analyze the feature vector, identifying abnormal traffic behavior and achieving network security monitoring.

[0064] Specifically, acquiring network traffic metadata is the first step in network security monitoring. This requires capturing data packets transmitted over the network and extracting key information such as timestamps, packet size, source IP address, destination IP address, source port number, and destination port number. This information constitutes the raw network traffic metadata and is stored in a database for subsequent analysis. For example, a data packet might be captured with a timestamp of October 27, 2024, 10:00:00, a packet size of 1500 bytes, a source IP address of 192.168.1.10, a destination IP address of 8.8.8.8, a source port number of 5555, and a destination port number of 80. To better analyze traffic data, the traffic metadata needs to be divided according to a preset time window size. For example, if a 1-minute time window is selected, all traffic data captured within that minute will be grouped into one time window. This allows traffic data to be grouped by time period, facilitating subsequent statistical analysis of each time period and revealing periodic patterns in traffic. For traffic data within each time window, the distribution of packet sizes can be statistically analyzed. For example, metrics such as average packet size, maximum packet size, and minimum packet size can be calculated. Suppose 1000 packets are captured within a time window, with an average size of 500 bytes, a maximum packet size of 1500 bytes, and a minimum packet size of 64 bytes. These metrics reflect network traffic load. For instance, a sudden increase in the average packet size might indicate a large amount of data transmission on the network, requiring further investigation. Besides packet size distribution characteristics, the arrival intervals of adjacent packets within each time window can be calculated, and the distribution of these time intervals can be statistically analyzed, such as average interval, maximum interval, and minimum interval. For example, a highly regular arrival interval between adjacent packets might indicate some periodic network activity, such as scheduled tasks or heartbeat packets. Conversely, a highly uneven distribution of time intervals might indicate sudden events or abnormal behavior on the network. Furthermore, determining the start and end times of each flow, calculating the flow duration, and statistically analyzing its distribution can also provide valuable information. A flow refers to a continuous sequence of data packets with the same source IP, destination IP, source port number, and destination port number. For example, a user visiting a website generates a flow. By analyzing the duration of the flow, we can understand how long the user visited the website and whether there are long-running connections. Long-running connections may indicate malware or a network attack. Combining the statistical features obtained from the above steps can create a high-dimensional feature vector reflecting traffic behavior patterns. For example, features such as average packet size, maximum packet size, average time interval, maximum time interval, and average flow duration within a time window can be combined into a feature vector. This feature vector can comprehensively describe the network traffic behavior patterns within that time window.Finally, machine learning algorithms, such as clustering or anomaly detection, can be used to analyze feature vectors and identify abnormal traffic behavior. For example, clustering algorithms can be used to group traffic data with similar characteristics, and then groups that differ significantly from other groups can be identified; these groups may represent abnormal traffic behavior. Similarly, anomaly detection algorithms can be used to identify data points that deviate significantly from normal traffic patterns; these data points may indicate network attacks or abnormal events, thereby achieving network security monitoring. In this way, potential network security threats can be detected and responded to in a timely manner, protecting network security.

[0065] S103. Use unsupervised machine learning algorithms, such as K-means clustering or autoencoders, to perform cluster analysis on traffic feature vectors, discover abnormal patterns and outliers in traffic, and determine potential malicious traffic clusters based on the clustering results.

[0066] This process involves acquiring network traffic data, extracting multi-dimensional feature vectors from the traffic, and constructing a feature matrix. The unsupervised learning algorithm K-means is then used to cluster the feature matrix, dividing the traffic into multiple clusters. The distance from each traffic sample to its cluster center is calculated; if the distance exceeds a preset threshold, it is identified as an anomaly. The proportion of anomalies within each cluster is statistically analyzed; if the proportion exceeds a preset threshold, the cluster is identified as a malicious traffic cluster. An autoencoder model is trained to reduce the dimensionality and reconstruct the traffic features, and the reconstruction error is calculated. If the reconstruction error of a traffic sample exceeds a preset threshold, it is identified as abnormal traffic. By combining the anomaly detection results from clustering and the autoencoder, the final malicious traffic is determined, achieving network security threat detection.

[0067] Specifically, network traffic data is acquired, for example, by capturing data packets using a network sniffer or extracting traffic information from network device logs. This data includes the timestamp, packet size, source IP address, destination IP address, source port, destination port, and protocol type of each data packet. Multidimensional feature vectors of the traffic are extracted, such as the average packet size, maximum packet size, minimum packet size, standard deviation of packet size, average, maximum, minimum, and standard deviation of the arrival time interval between adjacent packets within a time window, the duration of the flow, and the number of packets per second. Combining these features forms a multidimensional feature vector that describes the traffic behavior pattern. A feature matrix is ​​constructed by arranging the feature vectors of all traffic samples row-wise. Each row of this matrix represents a traffic sample, and each column represents a feature. The unsupervised learning algorithm K-means is used to cluster the feature matrix, for example, dividing the traffic into five clusters. The core idea of ​​the K-means algorithm is to group similar traffic samples into the same cluster and dissimilar traffic samples into different clusters. Suppose the center point of a cluster has coordinates (100, 5, 2, 10), where 100 represents an average packet size of 100 bytes, 5 represents an average packet arrival time interval of 5 milliseconds, 2 represents a flow duration of 2 seconds, and 10 represents 10 packets per second. A traffic sample's feature vector is (98, 6, 1.9, 11). Since it is close to the cluster center, it can be assigned to this cluster. The distance from each traffic sample to its cluster center is calculated. If the distance exceeds a preset threshold, it is considered an anomaly. For example, if the preset threshold is 3, and the distance from a traffic sample to its cluster center is 4, then that traffic sample is considered an anomaly. This indicates that the behavior pattern of this traffic sample differs significantly from other traffic samples in its cluster, potentially indicating abnormal behavior. The proportion of anomalies within each cluster is calculated. If the proportion exceeds a preset threshold, the cluster is considered a malicious traffic cluster. For example, if the preset threshold is 0.1. If a cluster contains 100 traffic samples, and 15 of them are outliers, then the outlier rate for that cluster is 0.15, exceeding the preset threshold of 0.1. Therefore, this cluster can be identified as a malicious traffic cluster, indicating that the traffic samples within it exhibit abnormal behavior and may contain malicious traffic. An autoencoder model, such as a three-layer autoencoder, is trained to reduce the dimensionality and reconstruct the traffic features. An autoencoder is a neural network model that compresses high-dimensional data into a low-dimensional representation and then restores the low-dimensional representation to high-dimensional data. By comparing the differences between the original data and the reconstructed data, the presence of anomalies can be determined. The reconstruction error is calculated. If the reconstruction error of a traffic sample exceeds a preset threshold, it is considered abnormal traffic. For example, the preset threshold is 0.05.The original feature vector of one traffic sample is (100, 5, 2, 10). After reconstruction by an autoencoder, the feature vector is (98, 6, 1.9, 11), with a reconstruction error of 0.03, which is less than a preset threshold. Therefore, this traffic sample is determined to be normal traffic. Another traffic sample has an original feature vector of (500, 1, 10, 100), and the reconstructed feature vector is (100, 5, 2, 10), with a reconstruction error of 0.8, which is greater than a preset threshold. Therefore, this traffic sample is determined to be abnormal traffic. By combining the anomaly detection results of clustering and autoencoder, the final malicious traffic is determined, achieving network security threat detection. For example, if a traffic sample is identified as an anomaly by the K-means algorithm, and its reconstruction error also exceeds a preset threshold, the traffic sample can be more confidently determined to be malicious traffic. Thus, by combining two different anomaly detection methods, the accuracy and reliability of detection can be improved.

[0068] S104. For suspected malicious traffic clusters, extract representative samples and match the sample traffic with the known malicious traffic feature database through the threat intelligence interface. If the matching degree exceeds the preset threshold, it is determined to be malicious traffic of a known type.

[0069] Data on suspected malicious traffic clusters is acquired, and clustering algorithms are used to analyze these clusters, obtaining the centroids of each cluster as representative samples. The extracted representative sample data is then sent to the threat intelligence platform via a threat intelligence interface. Upon receiving the representative sample data, the threat intelligence platform uses a similarity matching algorithm to calculate the similarity between the sample and data in the known malicious traffic feature database. Based on the similarity value calculated by the similarity matching algorithm, it is determined whether the similarity exceeds a preset similarity threshold. If the similarity exceeds the preset threshold, the traffic cluster containing the representative sample is classified as known malicious traffic of the corresponding type. If the similarity does not exceed the preset threshold, the traffic cluster containing the representative sample is temporarily marked as suspected malicious traffic of an unknown type, and relevant information is recorded for further analysis. For traffic clusters classified as known malicious traffic, corresponding security protection measures are implemented according to the type of malicious activity. For traffic clusters classified as suspected malicious traffic of unknown type, their behavioral characteristics are continuously monitored and analyzed, and the threat intelligence database is updated regularly.

[0070] Specifically, data on suspected malicious traffic clusters is acquired. For example, network traffic data is divided into multiple clusters using the K-means clustering algorithm, and several clusters suspected of containing malicious traffic are identified based on indicators such as the proportion of outliers within each cluster. The data of these identified clusters are the targets of the next step of analysis, for example, cluster A, cluster B, and cluster C. Clustering algorithms are used to analyze the traffic clusters, obtaining the centroid of each cluster as a representative sample. For the previously acquired suspected malicious traffic clusters A, B, and C, clustering algorithms, such as the K-medoids algorithm, are applied again to calculate the centroid of each cluster, serving as the representative sample for that cluster. Assume the feature vector of the centroid of cluster A is (10, 20, 30), cluster B is (5, 10, 15), and cluster C is (1, 2, 3). These feature vectors can represent the overall characteristics of the corresponding clusters. Selecting the centroid as the representative sample effectively reduces the computational load of subsequent analysis and captures the main characteristics of the cluster. The extracted representative sample data is then sent to the threat intelligence platform through a threat intelligence interface. The three representative sample feature vectors (10,20,30), (5,10,15), and (1,2,3) obtained above are sent to the threat intelligence platform in a standardized data format (e.g., JSON). This interface can be a REST API or other type of interface to ensure that the data can be correctly received and parsed by the platform. After receiving the representative sample data, the threat intelligence platform calls a similarity matching algorithm to calculate the similarity between the sample and the data in the known malicious traffic feature database. Assume that the threat intelligence platform's feature database stores a large number of known malicious traffic feature vectors; for example, the feature vector for a DDoS attack is (11,22,33), the feature vector for a port scan is (6,12,18), and the feature vector for normal traffic is (1,1,1). The platform uses a similarity matching algorithm, such as cosine similarity or Euclidean distance, to calculate the similarity between the representative sample and each sample in the feature database. Based on the similarity value calculated by the similarity matching algorithm, it is determined whether it exceeds a preset similarity threshold. Assume the preset similarity threshold is 0.9. The similarity calculation result between the representative sample and the samples in the feature database is compared with the threshold. For example, the similarity between the representative sample (10,20,30) of cluster A and the DDoS attack feature vector (11,22,33) is 0.95, exceeding the threshold. The similarity between the representative sample (5,10,15) of cluster B and the port scan feature vector (6,12,18) is 0.92, also exceeding the threshold. The similarity between the representative sample (1,2,3) of cluster C and the normal traffic feature vector (1,1,1) is 0.98, exceeding the threshold. If the similarity value exceeds the preset threshold, the traffic cluster containing that representative sample is classified as known malicious traffic of the corresponding type. Based on the above calculations, cluster A is classified as DDoS attack traffic, cluster B as port scan traffic, and cluster C as normal traffic.This approach allows for the rapid identification of known types of malicious traffic and provides a basis for subsequent security measures. If the similarity value does not exceed a preset threshold, the traffic cluster containing the representative sample is temporarily marked as suspected malicious traffic of an unknown type, and relevant information is recorded for further analysis. For example, if a representative sample in cluster D has a similarity of less than 0.9 with all samples in the feature database, then cluster D will be marked as suspected malicious traffic of an unknown type. For traffic clusters identified as known malicious traffic, corresponding security measures are taken based on the type of malicious activity. For instance, for cluster A identified as DDoS attack traffic, measures such as rate limiting and blocking the source IP can be implemented. For cluster B identified as port scanning traffic, port security policies can be strengthened, and unnecessary ports can be closed. These targeted protective measures can effectively reduce network security risks. For traffic clusters identified as suspected malicious traffic of unknown type, their behavioral characteristics are continuously monitored and analyzed, and the threat intelligence database is updated regularly.

[0071] S105. For suspected traffic clusters that do not match known malicious features, a supervised machine learning algorithm, such as support vector machine or decision tree, is used to construct a malicious traffic classification model. The model is then trained and optimized using traffic samples labeled by experts to improve the classification accuracy of the model.

[0072] For unknown malicious traffic clusters, expert-annotated malicious traffic sample data is obtained, and features are extracted from the traffic sample data to obtain traffic feature vectors. Based on the traffic feature vectors, a malicious traffic classification model is constructed using the Support Vector Machine (SVM) algorithm, and the SVM classifier is obtained through model training. Based on the traffic feature vectors, a malicious traffic classification model is constructed using the Decision Tree (DBT) algorithm, and the DBT classifier is obtained through model training. Unknown traffic data to be classified is obtained, and traffic features are extracted to obtain traffic feature vectors to be classified. The traffic feature vectors to be classified are input into the SVM classifier, and the classifier predicts the malicious probability of the traffic. The traffic feature vectors to be classified are input into the DBT classifier, and the classifier predicts the malicious probability of the traffic. Based on the malicious probabilities predicted by the SVM classifier and the DBT classifier, a weighted average is used to obtain the final malicious probability. If the final malicious probability exceeds a preset threshold, the unknown traffic is determined to be malicious traffic; otherwise, it is determined to be normal traffic.

[0073] Specifically, the identification and classification of unknown malicious traffic is a crucial aspect of network security. To address the ever-changing nature of malicious traffic, an effective classification model is needed. This study employs two algorithms—Support Vector Machine (SVM) and Decision Tree—to build the model, combining expert-annotated data for training and prediction. First, malicious traffic sample data labeled by security experts needs to be obtained. This sample data includes various types of malicious traffic, such as DDoS attack traffic, worm virus traffic, and port scanning traffic. Assume 1000 labeled malicious traffic samples and 1000 normal traffic samples are obtained. Next, feature extraction is performed on these traffic sample data. Commonly used traffic features include source IP address, destination IP address, port number, protocol type, packet size, number of packets, and time interval. For example, the feature vector of a malicious traffic sample can be extracted as: (192.168.1.10, 192.168.1.20, 80, TCP, 1024, 100, 0.1). Then, using the extracted traffic feature vectors, a malicious traffic classification model is constructed using both the SVM and Decision Tree algorithms. The core idea of ​​the Support Vector Machine (SVM) algorithm is to find an optimal hyperplane that separates samples of different classes as much as possible. The Decision Tree (DBT) algorithm, on the other hand, constructs a tree structure and progressively divides samples based on feature values ​​to achieve classification. Through training, SVM and DBT classifiers are obtained. Suppose we have unknown traffic data to be classified, and its feature vector is extracted as: (192.168.1.50, 192.168.1.100, 443, HTTPS, 2048, 50, 0.2). This feature vector is then input into both the SVM and DBT classifiers. The SVM classifier predicts a malicious probability of 0.8 based on the distance between the feature vector and the hyperplane. The DBT classifier predicts a malicious probability of 0.7 based on the path of the feature value within the tree structure. Finally, the malicious probabilities predicted by the two classifiers are weighted and averaged. Assume the weight of the SVM classifier is 0.6 and the weight of the DBT classifier is 0.4. The final malicious probability is: 0.8 * 0.6 + 0.7 * 0.4 = 0.76. Assume the preset malicious probability threshold is 0.75. Since the final malicious probability of 0.76 exceeds the threshold, the unknown traffic is classified as malicious. If the final malicious probability is below the threshold, it is classified as normal traffic. Using two classifiers combined with a weighted average can improve the accuracy and robustness of malicious traffic identification. This is because a single classifier may have biases, while combining the results of multiple classifiers can effectively reduce these biases.

[0074] S106. Deploy the trained malicious traffic classification model into the online detection module of the firewall to predict and classify real-time network traffic. If it is determined to be malicious traffic, trigger the firewall's blocking and isolation policies to intercept the malicious traffic in real time.

[0075] The system acquires network traffic data, extracts features, and vectorizes the data to obtain traffic feature vectors. These feature vectors are then input into a pre-trained malicious traffic classification model, which predicts the probability of the traffic being malicious. The system checks if the probability exceeds a preset malicious threshold; if it does, the traffic is classified as malicious, otherwise as normal. For malicious traffic, the firewall's blocking policy is triggered, blocking both the source and destination IP addresses in real time to prevent further propagation. Simultaneously, the malicious traffic is redirected to an isolated area for in-depth analysis, extracting key features and attack methods. Based on the analyzed malicious traffic features, the malicious traffic classification model is incrementally trained and optimized to improve its ability to detect and identify unknown malicious traffic. The optimized model is then redeployed to the firewall's real-time detection module for continuous iteration and updates, enabling dynamic detection and real-time defense against malicious traffic.

[0076] Specifically, network traffic data is like a rushing river, containing both normal communication data and malicious attack traffic. To protect network security, effective detection and defense against this traffic are necessary. Acquiring network traffic data is like setting up a monitoring point on the river to collect all the traffic information. This can be achieved through techniques such as network sniffers and traffic mirroring. For example, using Wireshark software to capture all data packets on a network interface, or configuring a switch to mirror port traffic to a dedicated monitoring server. Feature extraction and vectorization of traffic data is like extracting various indicators from the collected river water, such as color, turbidity, and flow rate, and quantifying these indicators into specific values ​​to form a feature vector. Features that can be extracted from network traffic include source IP address, destination IP address, port number, protocol type, packet size, packet frequency, and so on. For example, the feature vector of traffic accessing a web server can be represented as: [Source IP: 192.168.1.10, Destination IP: 202.100.1.100, Port Number: 80, Protocol Type: TCP, Packet Size: Average 500 bytes, Packet Frequency: 10 packets per second]. Quantizing these features into numerical vectors facilitates processing by subsequent machine learning models. Inputting the traffic feature vectors into a pre-trained malicious traffic classification model is like inputting extracted river water indicators into a water quality analyzer; the analyzer determines the safety of the water quality based on a pre-set model. The malicious traffic classification model can be built based on algorithms such as support vector machines, decision trees, and deep learning, and trained using a large number of known malicious traffic samples and normal traffic samples. S107: Continuously collect malicious traffic samples identified and blocked by the firewall. Through feature selection and fusion algorithms, automatically optimize the malicious traffic feature library and dynamically update the firewall's detection model and rule base, improving the ability to identify unknown and variant malicious traffic, and achieving adaptive evolution of the firewall.

[0077] We continuously collect malicious traffic samples blocked by the firewall, extract multi-dimensional feature vectors from the traffic, and construct a malicious traffic feature library. We employ unsupervised learning algorithms to cluster the malicious traffic feature library, automatically discovering new malicious traffic patterns and optimizing feature selection. We combine multiple features using feature fusion algorithms to generate high-dimensional composite features, improving the accuracy of malicious traffic identification. Based on the optimized feature library, we dynamically generate detection rules and update the firewall's rule base, achieving adaptive adjustment of the detection model. We perform online classification and judgment on suspicious traffic identified by the firewall; if the confidence level exceeds a threshold, we block it, and the samples are fed back for optimization. We continuously track the evolution of malicious traffic variants and dynamically expand the feature library through incremental learning methods, improving the detection rate of unknown malicious traffic. We comprehensively utilize rule matching, machine learning, and other methods to construct a multi-layered detection model, forming a defense-in-depth approach and enhancing the overall protection capability of the firewall.

[0078] Specifically, continuously collecting samples of malicious traffic blocked by firewalls and building a malicious traffic signature database is crucial. For example, a firewall might block traffic from an external IP address attempting to access an internal server. This traffic uses an illegal port number and has an abnormal packet length. This information, including source IP, destination IP, port number, protocol type, packet length, and timestamp, is recorded, converted into feature vectors, and stored in the malicious traffic signature database. This provides a sufficient data foundation for subsequent analysis and learning. Using unsupervised learning algorithms to cluster the malicious traffic signature database can automatically discover new malicious traffic patterns. For example, the K-Means algorithm can be used to cluster the collected malicious traffic feature vectors. Suppose the samples in the signature database are clustered into three categories: the first category is scanning attacks, characterized by randomly changing destination ports and high access frequency; the second category is DDoS attacks, characterized by numerous source IPs, a single destination port, and consistent packet size; and the third category is exploit attacks, characterized by payloads containing specific exploit code. Cluster analysis can uncover new malicious traffic patterns, such as a new type of DDoS attack where packet sizes are not uniform but exhibit a specific regularity. This helps in the timely detection and response to new attack methods, optimize feature selection, and thus improve the model's detection capabilities. Feature fusion algorithms can improve the accuracy of malicious traffic identification. For example, the features "source IP access frequency" and "destination port access frequency" can be merged into a new feature—"IP-port access frequency." If a source IP frequently accesses multiple different destination ports within a short period, or if a destination port is frequently accessed by multiple different source IPs, this may indicate malicious behavior, such as port scanning or DDoS attacks. Feature fusion integrates information from multiple features, thereby improving the accuracy of malicious traffic identification. Based on the optimized feature library, detection rules are dynamically generated, updating the firewall's rule base and enabling adaptive adjustments to the detection model. For example, analysis may reveal that some malicious traffic is characterized by a source IP address range within a specific range and access to port 8080. A new firewall rule can be generated based on this characteristic: "Block traffic from IP address range X.X.X.0 / 24, accessing port number 8080." When new malicious traffic patterns emerge, new detection rules can be dynamically generated and the firewall's rule base updated, enabling the firewall to adapt to new threat environments. Suspicious traffic identified by the firewall is classified and judged online; if the confidence level exceeds a threshold, it is blocked. For example, the firewall identifies traffic accessing a sensitive server; some characteristics of this traffic are similar to known malicious traffic, but the confidence level is only 70%. If the preset threshold is 80%, this traffic will not be immediately blocked but will be marked as suspicious traffic for further analysis.If subsequent analysis reveals malicious activity in the traffic, it will be blocked, and the sample will be used to optimize the model. This approach avoids false positives and allows for the collection of more data to improve model performance. Continuously tracking the evolution of malicious traffic variants and dynamically expanding the feature library through incremental learning improves the detection rate of unknown malicious traffic. For example, a known DDoS attack is characterized by the use of the UDP protocol and a fixed packet size. However, attackers may mutate it, using the TCP protocol or changing the packet size. By continuously tracking the evolution of malicious traffic, adding new variant samples to the feature library, and updating the model using incremental learning, the detection rate of unknown malicious traffic can be improved, thus better addressing evolving threats. A multi-layered detection model is constructed by comprehensively utilizing rule matching, machine learning, and other methods to form a defense-in-depth approach, enhancing the overall protection capabilities of the firewall. For example, the first layer uses rule matching to quickly filter out known malicious traffic; the second layer uses machine learning models for more refined analysis and judgment of suspicious traffic; and the third layer uses sandboxing technology to simulate the execution of highly suspicious traffic to further confirm whether it exhibits malicious behavior. This multi-layered defense system can effectively improve the overall protection capability of the firewall and resist various types of malicious traffic attacks.

[0079] On the other hand, this embodiment also provides a firewall-based network security detection system, including:

[0080] The data acquisition module is used to acquire network traffic data packets, parse the key fields of the data packets, and obtain structured traffic metadata.

[0081] The processing module is used to perform feature engineering on the structured traffic metadata to generate feature vectors that reflect traffic behavior patterns.

[0082] The suspected malicious traffic cluster acquisition module is used to perform cluster analysis on the feature vectors reflecting traffic behavior patterns based on unsupervised machine learning algorithms, and determine suspected malicious traffic clusters based on the clustering results.

[0083] The determination module is used to extract representative samples of the suspected malicious traffic clusters, and perform similarity matching between the sample traffic and the known malicious traffic feature library through the threat intelligence interface. If the matching degree exceeds a preset threshold, it is determined to be malicious traffic of a known type.

[0084] The model building module is used to train the support vector machine model and decision tree using traffic samples annotated by experts to obtain a malicious traffic classification model.

[0085] The detection module is used to feed the known types of malicious traffic as labels into the malicious traffic classification model for optimization, and to detect network security based on the optimized model.

[0086] On the other hand, this embodiment also provides an electronic device, including a memory, a processor, and a computing program stored in the memory and executable on the processor, wherein the processor implements the firewall-based network security detection method when executing the computer program.

[0087] On the other hand, this embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the firewall-based network security detection method.

[0088] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A firewall-based network security detection method, characterized by, The application relates to a network security detection method and device. The method comprises the following steps: Obtaining network traffic data packets, parsing key fields of the data packets, and obtaining structured traffic metadata; Performing feature engineering processing on the structured traffic metadata to generate feature vectors reflecting traffic behavior patterns; Performing clustering analysis on the feature vectors reflecting traffic behavior patterns based on an unsupervised machine learning algorithm, and determining a suspected malicious traffic cluster according to the clustering results; Extracting a representative sample of the suspected malicious traffic cluster, matching the sample traffic with a known malicious traffic feature library through a threat intelligence interface, and determining the malicious traffic as a known type of malicious traffic if the matching degree exceeds a preset threshold; Training a support vector machine model and a decision tree through expert-labeled traffic samples to obtain a malicious traffic classification model; Optimizing the known type of malicious traffic as a label in the malicious traffic classification model, and detecting network security based on the optimized model; The process of determining a suspected malicious traffic cluster according to the clustering results comprises the following steps: Clustering the feature vectors reflecting traffic behavior patterns through a K-means algorithm, and dividing the traffic into multiple clusters; Calculating the distance of each traffic sample to the cluster center to which the traffic sample belongs, and determining an abnormal point if the distance exceeds a preset threshold; Counting the proportion of abnormal points in each cluster, and determining the cluster as a malicious traffic cluster if the proportion exceeds a preset threshold; Training an autoencoder model to reduce and reconstruct the traffic features, calculating the reconstruction error, and determining a suspected malicious traffic cluster if the reconstruction error of the traffic sample exceeds a preset threshold; The process of detecting network security based on the optimized model comprises the following steps: Obtaining network traffic data, performing feature extraction and vectorization on the traffic data, and obtaining traffic feature vectors; Inputting the traffic feature vectors into the optimized model, and obtaining a malicious probability value of the traffic through model prediction; Determining whether the malicious probability value exceeds a preset malicious threshold, and determining the traffic as malicious traffic if the threshold is exceeded, otherwise determining the traffic as normal traffic; Triggering a blocking strategy of a firewall for the malicious traffic, blocking the source IP address and the destination IP address of the malicious traffic in real time, and guiding the malicious traffic to an isolation area to analyze the malicious traffic in depth and extract key features and attack means of the malicious traffic; 2. The method of claim 1, wherein, Incrementally training and optimizing the malicious traffic classification model according to the malicious traffic features obtained through analysis and the known type of malicious traffic, redeploying the optimized malicious traffic classification model to a real-time detection module of the firewall, and iteratively updating the model to realize dynamic detection and real-time defense of malicious traffic. The process of obtaining structured traffic metadata comprises the following steps: Using a deep packet inspection technology to parse captured network data packets and extract key field information; Processing the key field information to obtain the structured traffic metadata; wherein if the protocol type is TCP, the state and abnormality of network connection are determined according to the source IP, destination IP and port number information; if the protocol type is HTTP, HTTP request and response data are parsed, and URL, request method and status code are extracted.

3. The method of claim 1, wherein, The process of feature engineering on the structured traffic metadata to generate a feature vector reflecting a traffic behavior pattern comprises: dividing the structured traffic metadata according to a preset time window size to obtain time windows; obtaining a packet size distribution feature by counting the distribution of packet sizes in each time window according to the traffic data; obtaining a packet arrival time interval distribution feature by counting the distribution of time intervals between the arrival times of adjacent packets in each time window; judging the start and end times of each flow, calculating the duration of the flow, and counting the distribution of the duration of the flow to obtain a flow duration feature; combining the packet size distribution feature, the packet arrival time interval distribution feature, and the flow duration feature to obtain the feature vector reflecting the traffic behavior pattern.

4. The method of claim 1, wherein, The process of similarity matching between the sample traffic and the known malicious traffic feature library comprises: obtaining suspected malicious traffic cluster data, analyzing the traffic cluster, and obtaining the center point of each traffic cluster as a representative sample; sending the extracted representative sample data to a threat intelligence platform through a threat intelligence interface; after the threat intelligence platform receives the representative sample data, calling a similarity matching algorithm to calculate the similarity between the sample and the data in the known malicious traffic feature library; judging whether the similarity value calculated by the similarity matching algorithm exceeds a preset similarity threshold; if the similarity value exceeds the preset threshold, determining that the traffic cluster where the representative sample is located is the known malicious traffic of the corresponding type.

5. The method of claim 1, wherein, The process of training the support vector machine model and the decision tree through the expert-labeled traffic sample to obtain a malicious traffic classification model comprises: obtaining expert-labeled malicious traffic sample data, extracting features from the traffic sample data to obtain a traffic feature vector, constructing a malicious traffic classification model using a support vector machine algorithm according to the traffic feature vector, and obtaining a support vector machine classifier through model training; constructing a malicious traffic classification model using a decision tree algorithm according to the traffic feature vector, and obtaining a decision tree classifier through model training; generating the malicious traffic classification model based on the support vector machine classifier and the decision tree classifier.

6. A firewall-based network security detection system for implementing the method of any one of claims 1-5, characterized by It comprises: a data acquisition module for acquiring network traffic data packets, parsing key fields of the data packets, and obtaining structured traffic metadata; a processing module for feature engineering on the structured traffic metadata to generate a feature vector reflecting a traffic behavior pattern; a suspected malicious traffic cluster acquisition module for clustering analysis on the feature vector reflecting the traffic behavior pattern based on an unsupervised machine learning algorithm, and determining suspected malicious traffic clusters according to the clustering results; a determination module for extracting representative samples of the suspected malicious traffic clusters, matching the sample traffic with a known malicious traffic feature library through a threat intelligence interface, and determining the malicious traffic of a known type if the matching degree exceeds a preset threshold; a model construction module for training a support vector machine model and a decision tree through expert-labeled traffic samples to obtain a malicious traffic classification model; The detection module is configured to feed back the known type of malicious traffic as a label to the malicious traffic classification model for optimization, and detect network security based on the optimized model.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor implements the method of any one of claims 1-5 when executing the computer program.

8. A computer-readable storage medium storing a computer program, the computer-readable storage medium comprising: The computer program is executed by the processor to implement the method of any one of claims 1-5.

Images