A network malicious traffic detection and analysis method and system

Abstract

The present invention provides a network malicious traffic detection and analysis method and system, the method is to obtain suspected malicious traffic slice data of network traffic data according to a network traffic feature library, perform attack mode association analysis based on the abnormal value detection result of the suspected malicious traffic slice data to obtain the abnormal threat level, automatically generate the initial traffic management strategy based on a preset reasoning engine, and then perform noise reduction on the suspected malicious traffic slice data and feature extraction based on feature engineering technology to obtain multi-dimensional feature data, input the malicious traffic detection model to perform malicious traffic identification to obtain suspected zero-day attack samples, and obtain the malicious traffic prevention and control strategy according to the zero-day attack type of the suspected zero-day attack sample, which is used to adjust the initial traffic management strategy to generate the optimal traffic management strategy. The present invention can adaptively perceive the changes in traffic patterns and attack methods, deeply mine hidden attack patterns, and realize rapid and accurate identification and blocking of zero-day attacks.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and in particular to a method and system for detecting and analyzing network malicious traffic. Background Art

[0002] With the development of computer network technology, network security issues have received increasing attention, and the detection and prevention of malicious traffic is the main task of network security protection to avoid traffic access that attempts to gain unauthorized access or damage network resources.

[0003] Existing network malicious traffic detection methods mainly rely on statistical analysis and pattern recognition. Among them, the anomaly detection method based on machine learning has become a widely used solution for malicious traffic detection. However, existing machine learning detection methods usually use network models constructed by training malicious traffic data based on known attack types for anomaly identification and do not consider adaptive optimization based on identification errors, resulting in the inability to adapt to the ever-changing attack methods and patterns. In complex network environments where malicious traffic is diverse, concealed and deceptive, it is easy to generate a high number of false alarms and missed alarms, making it difficult to effectively ensure the accuracy and efficiency of malicious traffic detection. Faced with increasingly complex and diverse network attack scenarios, it is impossible to quickly and accurately discover and block unknown zero-day attacks to reliably ensure network security. Summary of the invention

[0004] The purpose of the present invention is to provide a network malicious traffic detection and analysis method, which generates an initial traffic management strategy by performing outlier detection on suspected malicious traffic slice data obtained based on abnormal slice analysis of real-time network traffic data, and combines the results obtained by mining hidden attack patterns of suspected malicious traffic slice data using deep learning technology to generate a malicious traffic prevention and control strategy, so as to generate an optimal traffic management strategy for real-time protection of network malicious traffic, which can adaptively perceive changes in traffic patterns and attack methods in complex network environments, deeply mine hidden attack patterns, and quickly and accurately identify and block zero-day attacks, thereby providing reliable protection for network security defense.

[0005] In order to achieve the above objectives, it is necessary to provide a network malicious traffic detection and analysis method and system in response to the above technical problems.

[0006] In a first aspect, an embodiment of the present invention provides a method for detecting and analyzing network malicious traffic, the method comprising the following steps:

[0007] Acquire network traffic data in real time, and acquire suspected malicious traffic slice data of the network traffic data based on a pre-built network traffic feature library;

[0008] Perform outlier detection on the suspected malicious traffic slice data, perform attack mode association analysis based on the corresponding outlier detection results, obtain the abnormal threat level, and automatically generate a corresponding initial traffic management strategy based on a preset reasoning engine according to the abnormal threat level;

[0009] Performing noise reduction processing on the suspected malicious traffic slice data to obtain corresponding noise reduction slice data, and performing multi-dimensional feature extraction on the noise reduction slice data based on a preset feature engineering technology to obtain corresponding multi-dimensional feature data; the multi-dimensional feature data includes time dimension feature data, space dimension feature data and semantic space feature data;

[0010] Inputting the multi-dimensional feature data into a pre-built malicious traffic detection model to identify malicious traffic, obtain suspected zero-day attack samples, and identify attack types based on the suspected zero-day attack samples to obtain corresponding zero-day attack types;

[0011] According to the zero-day attack type, a corresponding malicious traffic prevention and control strategy is obtained, and the malicious traffic prevention and control strategy and the initial traffic management strategy are comprehensively analyzed to generate an optimal traffic management strategy.

[0012] Furthermore, the suspected malicious traffic slice data includes the source IP address, destination port, payload content, payload size and transmission frequency of the data packet;

[0013] The step of obtaining suspected malicious traffic slice data of the network traffic data according to the pre-built network traffic feature library includes:

[0014] According to the network traffic feature library, extracting features from the network traffic data based on a statistical analysis method to obtain initial feature data;

[0015] According to a preset attention mechanism, the fusion weight of each feature data in the initial feature data is calculated based on the malicious traffic correlation, and the initial feature data is weightedly fused according to each fusion weight to obtain a corresponding high-dimensional feature vector;

[0016] The high-dimensional feature vector is subjected to dimensionality reduction processing and then input into a pre-built long short-term memory neural network for traffic classification to obtain a corresponding traffic classification result;

[0017] When the traffic classification result is suspected malicious traffic, a preset malicious traffic slicing operation is performed to extract corresponding suspected malicious traffic slicing data.

[0018] Furthermore, the step of performing outlier detection on the suspected malicious traffic slice data, performing attack mode association analysis according to the corresponding outlier detection result, and obtaining the abnormal threat level includes:

[0019] According to the preset outlier detection model, an outlier detection is performed on each slice feature data in the suspected malicious traffic slice data to obtain a corresponding outlier detection result;

[0020] Cluster analysis is performed on the slice feature data with all abnormal detection results marked as outliers to obtain multiple abnormal traffic patterns;

[0021] Based on the association rule mining technology, various abnormal traffic patterns are analyzed for correlation with known attack patterns to obtain corresponding correlation analysis results;

[0022] According to the correlation analysis result and the preset abnormal threat knowledge base, the corresponding abnormal threat level is obtained, and based on the online incremental learning mechanism, the preset abnormal value detection model and the preset abnormal threat knowledge base are continuously updated.

[0023] Furthermore, the step of performing noise reduction processing on the suspected malicious traffic slice data to obtain corresponding noise reduction slice data includes:

[0024] Performing noise type identification on the suspected malicious traffic slice data to obtain a corresponding noise type;

[0025] If the noise type is periodic stationary noise, the suspected malicious traffic slice data is subjected to frequency domain filtering processing by a preset bandpass filter to obtain the noise-reduced slice data; otherwise, a corresponding wavelet transform model is constructed according to a preset wavelet basis function and a decomposition layer number;

[0026] According to the wavelet transform model, multi-scale wavelet decomposition is performed on the suspected malicious traffic slice data to obtain corresponding wavelet coefficients;

[0027] According to a preset high-frequency threshold, the high-frequency components in the wavelet coefficients are identified, and the high-frequency components are set to zero to obtain corresponding denoised wavelet coefficients;

[0028] Using a wavelet reconstruction algorithm, the noise reduction wavelet coefficients are reconstructed into a time domain signal to obtain corresponding noise reduction feature data;

[0029] Calculating a signal-to-noise ratio of the noise reduction feature data, and obtaining a corresponding noise reduction performance evaluation result according to the signal-to-noise ratio;

[0030] When the noise reduction performance evaluation result reaches the preset noise reduction performance standard, the noise reduction feature data is used as the noise reduction slice data;

[0031] When the noise reduction performance evaluation result does not reach the preset noise reduction performance standard, the wavelet transform model and the preset high-frequency threshold are optimized, and the noise reduction feature data is subjected to noise reduction processing according to the optimized wavelet transform model and the preset high-frequency threshold until the preset noise reduction performance standard is reached, thereby obtaining the noise reduction slice data.

[0032] Furthermore, the malicious traffic detection model is obtained based on recursive neural network training; the recursive neural network includes an input layer, a multi-dimensional feature extraction module, a feature fusion module and a fully connected layer; the multi-dimensional feature extraction module includes a bidirectional long short-term memory network;

[0033] The training steps of the malicious traffic detection model include:

[0034] Acquire a historical network traffic data set, and process the historical network traffic data set to obtain a corresponding historical noise reduction slice data set;

[0035] According to the preset feature engineering technology, multi-dimensional feature extraction is performed on the historical denoised slice data set to obtain a corresponding historical multi-dimensional feature data set;

[0036] Performing cluster analysis on the historical multi-dimensional feature data set to obtain a plurality of typical malicious traffic patterns, and obtaining a malicious traffic prediction category set based on the typical malicious traffic patterns;

[0037] According to the historical multi-dimensional feature data set and the malicious traffic prediction category set, performing parameter optimization training on the recursive neural network to obtain an initial traffic detection model;

[0038] According to the knowledge distillation processing technology, the initial traffic detection model is compressed to obtain the malicious traffic detection model.

[0039] Furthermore, the step of inputting the multi-dimensional feature data into a pre-built malicious traffic detection model to identify malicious traffic and obtain suspected zero-day attack samples includes:

[0040] Inputting the multi-dimensional feature data into the multi-dimensional feature extraction module for feature extraction to obtain a multi-level feature representation; the multi-level feature representation includes time dimension features, space dimension features and semantic space features;

[0041] Inputting the multi-level feature representation into a feature fusion module for feature fusion processing to obtain a corresponding deep feature representation;

[0042] Inputting the deep feature representation into the fully connected layer for classification prediction to obtain corresponding malicious traffic identification results;

[0043] When the malicious traffic identification result is malicious traffic, the corresponding network traffic is marked as a suspected zero-day attack sample.

[0044] Furthermore, the step of identifying the attack type according to the suspected zero-day attack sample to obtain the corresponding zero-day attack type includes:

[0045] Obtaining a key feature vector corresponding to the suspected zero-day attack sample; the key feature vector includes a data packet flow feature, a transmission frequency feature, and a communication transmission feature;

[0046] Calculate the similarity between the key feature vector and each known attack type feature vector in the preset zero-day attack feature library by using the cosine similarity algorithm, and determine whether the average similarity is greater than a preset similarity threshold;

[0047] If so, the known attack type corresponding to the maximum similarity is used as the zero-day attack type of the suspected zero-day attack sample; otherwise, the suspected zero-day attack sample is determined to be a new type of zero-day attack, and the key feature vector of the suspected zero-day attack sample is added to the preset zero-day attack feature library.

[0048] Furthermore, the step of obtaining a corresponding malicious traffic prevention and control strategy according to the zero-day attack type includes:

[0049] If the zero-day attack type is a known attack type, a traffic security management policy corresponding to the known attack type is obtained from a preset attack pattern knowledge base, and the traffic security management policy is used as the malicious traffic prevention and control policy;

[0050] If the zero-day attack type is a new type of zero-day attack, the basic traffic characteristics of the suspected zero-day attack sample are evaluated for abnormality according to the preset isolation forest model, a corresponding attack analysis report is generated, and traffic prevention and control prediction is performed according to the attack analysis report and the preset reinforcement learning model to obtain the malicious traffic prevention and control strategy.

[0051] Furthermore, the method further comprises:

[0052] Continuously monitor the recognition error traffic data of the malicious traffic detection model, classify the recognition error patterns according to the recognition error traffic data, and adjust and update the malicious traffic detection model according to the corresponding recognition error patterns; the recognition error traffic data includes malicious false alarm traffic data and malicious missed alarm traffic data.

[0053] In a second aspect, an embodiment of the present invention provides a network malicious traffic detection and analysis system, the system comprising:

[0054] The traffic slice processing module is used to obtain network traffic data and obtain suspected malicious traffic slice data of the network traffic data according to a pre-built network traffic feature library;

[0055] An initial strategy generation module is used to perform outlier detection on the suspected malicious traffic slice data, perform attack mode association analysis based on the corresponding outlier detection results, obtain the abnormal threat level, and automatically generate a corresponding initial traffic management strategy based on a preset reasoning engine according to the abnormal threat level;

[0056] A multi-dimensional feature extraction module is used to perform noise reduction processing on the suspected malicious traffic slice data to obtain corresponding noise reduction slice data, and perform multi-dimensional feature extraction on the noise reduction slice data based on a preset feature engineering technology to obtain corresponding multi-dimensional feature data; the multi-dimensional feature data includes time dimension feature data, space dimension feature data and semantic space feature data;

[0057] A zero-day attack identification module is used to input the multi-dimensional feature data into a pre-built malicious traffic detection model to identify malicious traffic, obtain suspected zero-day attack samples, and identify attack types based on the suspected zero-day attack samples to obtain corresponding zero-day attack types;

[0058] The optimal strategy generation module is used to obtain the corresponding malicious traffic prevention and control strategy according to the zero-day attack type, and comprehensively analyze the malicious traffic prevention and control strategy and the initial traffic management strategy to generate the optimal traffic management strategy.

[0059] The present application provides a network malicious traffic detection and analysis method and system, through which the network traffic data is obtained in real time, suspected malicious traffic slice data of the network traffic data is obtained according to a pre-built network traffic feature library, and the attack mode association analysis is performed according to the abnormal value detection result of the suspected malicious traffic slice data to obtain the abnormal threat level, and according to the abnormal threat level, the corresponding initial traffic management strategy is automatically generated based on the preset reasoning engine, and the suspected malicious traffic slice data is subjected to noise reduction processing to obtain the corresponding noise reduction slice data, and the multi-dimensional feature data obtained by multi-dimensional feature extraction of the noise reduction slice data based on the preset feature engineering technology is input into the pre-built malicious traffic detection model to perform malicious traffic identification to obtain suspected zero-day attack samples and attack type identification to obtain the corresponding zero-day attack type, and the corresponding malicious traffic prevention and control strategy is obtained according to the zero-day attack type, and the initial traffic management strategy is adjusted according to the malicious traffic prevention and control strategy to generate the optimal traffic management strategy. Technical solution. Compared with the prior art, the network malicious traffic detection and analysis method can adaptively perceive the changes in traffic patterns and attack methods in complex network environments, deeply explore the hidden attack modes of malicious traffic, and quickly and accurately identify and block zero-day attacks, providing reliable protection for network security defense. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a flowchart of a method for detecting and analyzing network malicious traffic in an embodiment of the present invention;

[0061] Figure 2 It is another flow chart of the method for detecting and analyzing network malicious traffic in an embodiment of the present invention;

[0062] Figure 3 It is a structural diagram of a network malicious traffic detection and analysis system in an embodiment of the present invention;

[0063] Figure 4 It is another structural schematic diagram of the network malicious traffic detection and analysis system in an embodiment of the present invention. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical scheme and beneficial effects of the present application clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. Obviously, the embodiments described below are part of the embodiments of the present invention and are only used to illustrate the present invention, but are not used to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0065] The network malicious traffic detection and analysis method provided by the present invention can be understood as an application defect that is based on the fact that the existing network malicious traffic detection method cannot adapt to the ever-changing attack means and patterns, which makes it easy to produce high false alarms and missed alarms in a complex network environment where malicious traffic presents the characteristics of diversity, concealment and deception. Instead, a network malicious traffic real-time protection solution is proposed, which generates an initial traffic management strategy by performing an outlier detection on the suspected malicious traffic slice data obtained based on the abnormal slice analysis of real-time network traffic data, and generates a malicious traffic prevention and control strategy based on the result of mining the hidden attack pattern of the suspected malicious traffic slice data using deep learning technology. The following embodiments will explain the network malicious traffic detection and analysis method of the present invention in detail.

[0066] In one embodiment, Figure 1 As shown, a network malicious traffic detection and analysis method is provided, comprising the following steps:

[0067] S11. Obtain network traffic data in real time, and obtain suspected malicious traffic slice data of the network traffic data according to a pre-built network traffic feature library; wherein, network traffic data can be understood as raw data packets collected in real time from a network environment, including normal traffic and various potential normal malicious traffic. The collection process only needs to meet high throughput and low latency requirements to ensure the real-time and integrity of data traffic analysis. The specific collection method is not specifically limited here. The network traffic feature library in this embodiment can be understood as a database that can be used for feature extraction and analysis of network traffic data based on the analysis and processing of the collected historical network traffic data, including different traffic types and corresponding traffic feature indicators. Its construction process includes pre-processing of historical network traffic data including data cleaning and data labeling, wherein data cleaning includes removing duplicate data packets and filtering irrelevant data packets, and data labeling can be understood as packaging labels for each data packet to distinguish normal traffic from malicious traffic. The specific labeling method and the adopted labeling tool can be selected according to actual needs.

[0068] In actual applications, high-performance data collection equipment can be deployed in the network environment, such as network probes with 10Gbps throughput and microsecond latency, to capture data packets transmitted in the network in real time, and use hash algorithms to deduplicate data packets. In addition, a rule-based filtering mechanism is used to remove data packets that are not related to malicious traffic, such as normal HTTP requests and DNS queries. Machine learning algorithms are then used to automatically label data packets, distinguish normal traffic from potential malicious traffic, and build a network traffic feature library containing information such as traffic types and feature indicators to perform slice analysis on suspected malicious traffic in real-time network traffic.

[0069] The suspected malicious traffic slice data in this embodiment can be understood as the slice feature data of the possible malicious traffic extracted from the real-time network traffic, which is used to improve the efficiency and accuracy of the subsequent in-depth analysis of the traffic, and preferably includes traffic features such as the source IP address, destination port, load content, load size and transmission frequency of the data packet. Specifically, the step of obtaining the suspected malicious traffic slice data of the network traffic data according to the pre-built network traffic feature library includes:

[0070] According to the network traffic feature library, the network traffic data is feature extracted based on a statistical analysis method to obtain initial feature data; wherein the initial feature data includes key traffic data such as the size distribution of data packets, the transmission frequency, the source IP address and the destination IP address, wherein the size distribution of data packets can be estimated by a probability density function, the transmission frequency can be obtained by Fourier transform analysis, and the source IP address and the destination IP address can be obtained by communication mode analysis using a clustering algorithm, all of which can be implemented by referring to relevant existing technologies and will not be described in detail here.

[0071] According to the preset attention mechanism, the fusion weight of each feature data in the initial feature data is calculated based on the malicious traffic correlation, and the initial feature data is weightedly fused according to each fusion weight to obtain a corresponding high-dimensional feature vector; wherein, the fusion weight of each feature data can be obtained based on the correlation measurement (cosine similarity or Euclidean distance, etc.) between the corresponding data feature and the malicious traffic, or can be generated by learning a multi-layer perceptron MLP network, so as to give different features different weights and highlight the key feature data that plays an important role in malicious traffic discrimination, which helps to improve the accuracy and efficiency of subsequent malicious traffic identification and analysis; it should be noted that the preset attention mechanism in this embodiment can be implemented using an existing attention mechanism that can achieve the corresponding function, and is not specifically limited here.

[0072] The high-dimensional feature vector is subjected to dimensionality reduction processing and then input into a pre-built long short-term memory neural network for traffic classification to obtain the corresponding traffic classification result; wherein, the dimensionality reduction processing can be understood as processing the high-dimensional feature vector obtained above using a dimensionality reduction algorithm such as principal component analysis to obtain a low-dimensional feature vector and then input it into the long short-term memory neural network for traffic classification to eliminate redundant information and noise interference, so as to improve the reliability of traffic classification. The long short-term memory neural network in this embodiment can be understood as a parameter optimal model obtained by pre-training with relevant historical data, preferably including 3 hidden layers and 128 neurons in each layer. During the training process of the network model, in order to effectively avoid the risk of model performance degradation caused by data imbalance, this embodiment adopts a positive and negative sample balance strategy to reasonably adjust the ratio of normal traffic and malicious traffic to avoid overfitting or underfitting of the model, and continuously optimizes the parameters of the recursive neural network model through multiple iterative training, so that it can accurately identify malicious traffic while maintaining a low false alarm rate for normal traffic; for example, the ratio of malicious traffic to normal traffic is controlled at 1:10, and the model parameters are optimized by the cross-validation method, so that the malicious traffic detection rate of the model on the test set reaches more than 95%, and the normal traffic false alarm rate is controlled within 1%. In practical applications, the traffic classification results obtained by the long short-term memory neural network for traffic classification of the input low-dimensional feature vector include normal traffic and suspected malicious traffic. Normal traffic does not need to be analyzed again, while suspected malicious traffic needs to continue to be deeply analyzed through the following method steps to ensure the reliability of the malicious traffic analysis results.

[0073] When the traffic classification result is suspected malicious traffic, a preset malicious traffic slicing operation is performed to extract the corresponding suspected malicious traffic slicing data. The preset malicious traffic slicing operation can be understood as first determining the traffic range that needs to be sliced ​​(data segments directly related to malicious traffic) based on the analysis results of the suspected malicious traffic, then extracting the determined sliced ​​traffic range from the original traffic data, and then extracting the corresponding data feature information from the obtained sliced ​​data segment, that is, summarizing the traffic features including the source IP address, destination port, load content, load size and transmission frequency of the data packet to obtain the required suspected malicious traffic slicing data (suspected malicious traffic feature slicing); in addition, a structured analysis report can also be generated based on all the suspected malicious traffic slicing data obtained, and the report content includes multi-dimensional information such as the time distribution, geographical distribution and target resources of the malicious traffic, so as to provide reliable data support for related security protection and source tracing analysis.

[0074] This embodiment uses a pre-built network traffic feature library combined with a long short-term memory neural network to perform traffic classification and identification and perform slice analysis on the identified suspected malicious traffic. It can not only improve the efficiency of continuous analysis based on suspected malicious traffic slice data and shorten the analysis time, but also reduce the analysis interference of irrelevant data through in-depth analysis of the slice data, improve the accuracy of malicious traffic analysis, reduce false alarms, and effectively reduce the waste of data resources.

[0075] S12, performing outlier detection on the suspected malicious traffic slice data, performing attack pattern association analysis based on the corresponding outlier detection results, obtaining an abnormal threat level, and automatically generating a corresponding initial traffic management strategy based on a preset reasoning engine according to the abnormal threat level; wherein, the outlier detection can be understood as detecting whether each feature data in the suspected malicious traffic slice data is normal based on a predetermined threshold range of each feature value, so as to further analyze the severity of the corresponding abnormal threat based on the abnormal feature data; specifically, the step of performing outlier detection on the suspected malicious traffic slice data, performing attack pattern association analysis based on the corresponding outlier detection results, and obtaining the abnormal threat level includes:

[0076] According to the preset outlier detection model, anomaly detection is performed on each slice feature data in the suspected malicious traffic slice data to obtain a corresponding outlier detection result; wherein, the preset outlier detection model can be understood as a rule-based outlier detection algorithm model, such as using a one-class support vector machine (One-Class SVM, OC-SVM), which is used to determine whether each slice feature data in the suspected malicious traffic slice data exceeds the corresponding preset threshold range. If exceeded, it is marked as an outlier, that is, the corresponding outlier detection result is marked as an outlier.

[0077] All slice feature data marked as abnormal values ​​in the abnormal detection results are clustered and analyzed to obtain a variety of abnormal traffic patterns; the cluster analysis can use the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) clustering algorithm to cluster the abnormal slice feature data; for example, by setting the cluster radius (ε=5) and the minimum number of samples (MinPts=10), three abnormal traffic patterns such as DDoS attacks, port scanning and worm propagation can be identified. It should be noted that the above cluster analysis parameter settings and the types of abnormal traffic patterns identified are all exemplary descriptions, which will vary depending on the actual analysis scenario in actual applications and are not specifically limited here.

[0078] According to the association rule mining technology, various abnormal traffic patterns are analyzed for correlation with known attack patterns to obtain corresponding correlation analysis results; wherein, the association rule mining technology can use the Apriori association rule mining algorithm, after setting the minimum support and the minimum confidence, the Apriori association rule mining algorithm is executed to find whether there is a significant correlation between the abnormal traffic pattern obtained by the above cluster analysis and all currently known attack patterns; correspondingly, the known attack patterns may include SYN Flood attack, UDP Flood attack, ICMP Flood attack, HTTP Flood attack, CC (Challenge Collapsar) attack and DNS Amplification attack, etc. In practical applications, if the abnormal traffic pattern is a DDoS attack traffic with a high correlation with the known SYN Flood attack, the correlation analysis result of the abnormal traffic pattern of the DDoS attack is highly correlated with the known SYN Flood attack, and the abnormal threat level of the abnormal traffic pattern can be determined according to the threat level of the known SYN Flood attack.

[0079] According to the correlation analysis results and the preset abnormal threat knowledge base, the corresponding abnormal threat level is obtained, and based on the online incremental learning mechanism, the preset abnormal value detection model and the preset abnormal threat knowledge base are continuously updated; wherein, the preset abnormal threat knowledge base can be understood as a database established based on known attack patterns, and includes a plurality of different known attack patterns corresponding to threat levels; in actual applications, after obtaining the highly correlated attack patterns corresponding to each abnormal traffic pattern through the above method steps, the preset abnormal threat knowledge base can be queried according to the corresponding highly correlated attack pattern to obtain the corresponding threat degree under the attack pattern, and the abnormal threat level of the corresponding abnormal traffic pattern is determined accordingly to determine its potential impact on network security. For example, after analysis, it can be obtained that the abnormal threat level of DDoS attack traffic is high, the abnormal threat level of port scanning traffic is medium, and the abnormal threat level of worm propagation traffic is low.

[0080] After obtaining the abnormal threat level corresponding to various abnormal traffic patterns through the above method steps, the required initial traffic management strategy can be generated based on the preset reasoning engine in combination with the corresponding suspected malicious traffic slice data. For example, the security strategy is automatically generated based on the Drools reasoning engine: blocking operations are performed on DDoS attack traffic with high threat levels, speed limiting operations are performed on port scanning traffic with medium threat levels, and isolation operations are performed on worm propagation traffic with low threat levels. It should be noted that the preset abnormal value detection model and the preset abnormal threat knowledge base in this embodiment can be updated in real time or regularly through an online incremental learning mechanism, and the normal threshold range of each feature data in the preset abnormal value detection model is dynamically adjusted using an adaptive threshold algorithm to effectively adapt to the diversity of network environments and network attack methods, and ensure the reliability and stability of continuous analysis of network traffic. At the same time, visual tools such as Echarts can also be used to generate intuitive analysis reports of various abnormal traffic patterns, and by combining log analysis and data mining technology, the IP addresses, attack targets and attack methods of suspected malicious traffic can be deeply analyzed to reveal the geographical distribution, behavior patterns and behavior intentions of attackers, etc., to provide a reliable basis for subsequent security maintenance and improve the efficiency and operability of network security management.

[0081] S13, performing noise reduction processing on the suspected malicious traffic slice data to obtain corresponding noise reduction slice data, and performing multi-dimensional feature extraction on the noise reduction slice data based on a preset feature engineering technology to obtain corresponding multi-dimensional feature data; wherein the noise reduction slice data can be understood as data obtained by performing noise removal processing on the suspected malicious traffic slice data based on a preset noise reduction technology; in order to ensure that the detailed information of the original data can be effectively retained while removing the noise, this embodiment preferably uses wavelet transform technology to remove non-stationary noise; specifically, the step of performing noise reduction processing on the suspected malicious traffic slice data to obtain corresponding noise reduction slice data includes:

[0082] The noise type of the suspected malicious traffic slice data is identified to obtain the corresponding noise type; wherein, the noise type includes two types: periodic stationary noise and non-stationary noise. The corresponding noise type identification can be achieved by analysis based on methods such as variance analysis, autocorrelation analysis and distribution characteristics. The specific implementation steps can be referred to the relevant technical implementation, which will not be repeated here.

[0083] If the noise type is periodic stationary noise, the suspected malicious traffic slice data is subjected to frequency domain filtering processing by a preset bandpass filter to obtain the denoised slice data. Otherwise, the corresponding wavelet transform model is constructed according to the preset wavelet basis function and the number of decomposition layers. That is, when the noise carried in the suspected malicious traffic slice data has obvious frequency domain characteristics, a frequency domain filtering method can be used to design a suitable filter to perform frequency domain filtering processing on the suspected malicious traffic slice data. For example, the center frequency can be set to the noise main frequency of 50Hz and the bandwidth can be set to 10Hz to filter out high-frequency noise components and effectively filter out periodic interference to obtain denoised data (denoised slice data). When the noise carried in the suspected malicious traffic slice data is a non-stationary signal, an adaptive wavelet threshold denoising method is used to adaptively design a wavelet transform model according to the statistical characteristics of the noise to achieve effective suppression of non-stationary noise. In this embodiment, the preset wavelet basis function and the number of decomposition layers in the wavelet transform model can be selected according to the statistical characteristics of the suspected malicious traffic slice data in actual applications. For example, based on the characteristics of network traffic, the Daubechies4 wavelet is selected as the basis function, and the slice data is decomposed by 5 layers of wavelets to map the original data to the wavelet domain space.

[0084] According to the wavelet transform model, the suspected malicious traffic slice data is subjected to multi-scale wavelet decomposition to obtain corresponding wavelet coefficients; wherein, the specific method for obtaining the wavelet coefficients can refer to the existing multi-scale wavelet decomposition implementation technology, which will not be described in detail here.

[0085] According to the preset high-frequency threshold, the high-frequency components in the wavelet coefficients are identified, and the high-frequency components are set to zero to obtain the corresponding denoised wavelet coefficients; wherein, the preset high-frequency threshold can be adjusted and set according to the actual application requirements. In order to ensure the adaptive suppression of non-stationary noise, this embodiment is preferably adaptively determined based on the noise variance of the obtained wavelet coefficients of each layer, and the high-frequency component coefficients less than the threshold are set to zero.

[0086] The wavelet reconstruction algorithm is used to reconstruct the denoising wavelet coefficients in the time domain to obtain corresponding denoising feature data; wherein, the denoising feature data can in principle be directly used as data for subsequent analysis, but in order to ensure the adequacy of the denoising process, this embodiment preferably evaluates and analyzes the data denoising effect through the following denoising performance evaluation method, which is used as a basis for optimizing the wavelet basis function and the number of decomposition layers in the wavelet transform model.

[0087] Calculate the signal-to-noise ratio of the noise reduction feature data, and obtain the corresponding noise reduction performance evaluation result based on the signal-to-noise ratio; wherein the signal-to-noise ratio (SNR) is an important indicator for measuring signal quality. The higher the signal-to-noise ratio, the greater the proportion of useful information, and vice versa, it means that the distortion is serious. The noise reduction performance evaluation result determined based on the signal-to-noise ratio can be understood as the noise reduction performance level corresponding to the data value range interval in which the signal-to-noise ratio is located. The specific mapping relationship between the signal-to-noise ratio and the noise reduction performance level can be set according to the actual application requirements, and is not specifically limited here.

[0088] When the noise reduction performance evaluation result reaches the preset noise reduction performance standard, the noise reduction feature data is used as the noise reduction slice data; wherein, the preset noise reduction performance standard can be understood as a pre-set noise reduction performance benchmark level. If this level standard is reached, it indicates that the above-mentioned noise reduction processing is effective. Otherwise, it indicates that the noise reduction processing cannot meet the requirements and the noise reduction processing still needs to be continued.

[0089] When the noise reduction performance evaluation result does not meet the preset noise reduction performance standard, the wavelet transform model and the preset high-frequency threshold are optimized, and the noise reduction feature data is denoised according to the optimized wavelet transform model and the preset high-frequency threshold until the preset noise reduction performance standard is met to obtain the noise reduction slice data; it should be noted that the specific process of optimizing the wavelet transform model and the preset high-frequency threshold can be understood as firstly combining and verifying the candidate basis functions and the number of decomposition layers based on a fixed data set according to a pre-designed combination principle, and then evaluating and verifying based on different network traffic data sets, so as to effectively ensure that the noise reduction requirements can be met while having certain generalization performance, good robustness and adaptability, and being able to maintain the stability of the noise reduction processing effect under different noise environments.

[0090] In order to ensure the reliability and comprehensiveness of the subsequent in-depth analysis of malicious traffic, this embodiment preferably extracts multi-dimensional feature data including time dimension feature data, space dimension feature data and semantic space feature data based on preset feature engineering technology; wherein, the time dimension feature data can be understood as the timestamp information of the traffic data, and the specific acquisition process can be understood as converting the timestamp data in the traffic data packet to obtain the time data in the preset format, so as to facilitate the subsequent calculation of the time interval of the traffic data packet and obtain the timing characteristics based on the time series analysis method; the space dimension feature data can be understood as the spatial information such as the source IP address, destination IP address and port number of the traffic data, which can also be obtained by parsing the relevant traffic data packets; the semantic space feature data can be understood as the semantic information such as the protocol type and load content of the traffic data, which can be obtained by parsing the transport layer and application layer data of the relevant traffic data packets. It should be noted that the above multi-dimensional feature extraction based on the preset feature engineering technology also includes the removal of invalid values ​​and normalization of the relevant data, which will not be described in detail here.

[0091] S14. Input the multi-dimensional feature data into a pre-built malicious traffic detection model to identify malicious traffic and obtain suspected zero-day attack samples, and identify the attack type based on the suspected zero-day attack samples to obtain the corresponding zero-day attack type; wherein the malicious traffic detection model is obtained based on recursive neural network training; the recursive neural network preferably includes an input layer, a multi-dimensional feature extraction module, a feature fusion module and a fully connected layer; the multi-dimensional feature extraction module includes a bidirectional long short-term memory network (Bidirectional Long Short-Term Memory, BiLSTM), and the bidirectional long short-term memory network includes multiple hidden layers, and in order to prevent overfitting, Dropout regularization is used after each hidden layer, for example, a multi-dimensional feature extraction module including 3 layers of bidirectional long short-term memory network hidden layers is constructed, and the number of neurons in each layer is 128, 256 and 128 respectively, and the Dropout regularization method is used to prevent overfitting; the corresponding fully connected layer preferably includes 6 layers of fully connected neural networks to fit the classification labels of the recursive neural network.

[0092] Specifically, the training steps of the malicious traffic detection model include:

[0093] Obtain a historical network traffic data set, and process the historical network traffic data set to obtain a corresponding historical noise reduction slice data set; wherein, the acquisition process of the historical noise reduction slice data set can refer to the related description of the acquisition of suspected malicious traffic slice data mentioned above, which will not be repeated here.

[0094] According to the preset feature engineering technology, multi-dimensional features are extracted from the historical denoised slice data set to obtain a corresponding historical multi-dimensional feature data set; wherein the historical multi-dimensional feature data set also includes time dimension feature data, space dimension feature data and semantic space feature data. For the corresponding acquisition method, please refer to the relevant description of the above multi-dimensional feature data.

[0095] The historical multi-dimensional feature data set is clustered and analyzed to obtain multiple typical malicious traffic patterns, and a malicious traffic prediction category set is obtained based on the typical malicious traffic patterns; wherein the malicious traffic prediction category set can be understood as a collection of all typical malicious traffic patterns automatically obtained by clustering the historical multi-dimensional feature data set using a K-means clustering algorithm, and each typical malicious traffic pattern is used as a malicious traffic classification label in a candidate training traffic detection model.

[0096] According to the historical multi-dimensional feature data set and the malicious traffic prediction category set, the recursive neural network is trained with parameters for optimization to obtain an initial traffic detection model; wherein, the process of performing parameter optimization training on the recursive neural network can be understood as annotating the historical multi-dimensional feature data set based on the malicious traffic prediction category set, optimizing the model parameters of the recursive neural network based on the Adam optimizer, and minimizing the loss function through the back propagation algorithm during the model training process, so that the model can automatically learn the hidden patterns of malicious traffic; at the same time, during the model training process, hyperparameters such as the learning rate and the regularization coefficient are also dynamically adjusted to improve the convergence speed and generalization ability of the model. For example, a dynamic adjustment learning rate attenuation strategy is adopted to reduce the learning rate by a preset proportion (for example, 20%) every 10 epochs until the accuracy on the test set reaches 95% and the recall rate reaches more than 98%, and the iteration is stopped to obtain the required initial traffic detection model.

[0097] According to the knowledge distillation processing technology, the initial traffic detection model is compressed to obtain the malicious traffic detection model; wherein, the knowledge distillation (KD) processing technology can be understood as a model compression method. After compressing the initial traffic detection model, not only the number of malicious traffic detection model parameters is reduced by about 90% relative to the number of initial traffic detection model parameters, but also the computational overhead of the model can be effectively reduced, and the corresponding inference speed is also increased by 5 times; in addition, when the trained malicious traffic detection model is integrated into the Suricata intrusion detection system for real-time detection performance testing, the average detection delay of 10 million real-time network traffic data scenarios is less than 10ms, and 253 unknown malicious traffic events can be accurately detected, which can effectively meet the real-time detection needs of network traffic.

[0098] The malicious traffic detection model trained and constructed through the above method steps can be used to reliably detect and analyze network traffic data, while ensuring the real-time and efficiency of data traffic analysis, thereby providing reliable guarantees for the timeliness and accuracy of zero-day attack detection.

[0099] Specifically, the step of inputting the multi-dimensional feature data into a pre-built malicious traffic detection model to identify malicious traffic and obtain suspected zero-day attack samples includes:

[0100] The multi-dimensional feature data is input into the multi-dimensional feature extraction module for feature extraction to obtain a multi-level feature representation; wherein the multi-level feature representation can be understood as the multi-dimensional feature data obtained by extracting multi-dimensional features such as time, space and semantics from the multi-dimensional feature data passed into the input layer by the bidirectional long short-term memory network of the multi-dimensional feature extraction module; that is, the multi-level feature representation includes time dimension features, space dimension features and semantic space features, wherein the time dimension features include time series features such as duration, periodicity and burstiness of the flow, and may also include energy features at different frequency scales extracted by multi-scale decomposition of the flow time series using the wavelet transform method, as well as the calculated flow peak average The spatial dimension features include the geographical distribution, topological structure and node relationship of the traffic. The geographical distribution features of the traffic can be extracted by clustering the source IP address and the destination IP address, and the IP address association graph can be constructed by calculating the Euclidean distance, cosine similarity and other indicators between the IP addresses to mine the spatial features of the traffic such as the topological structure and node relationship. The semantic space features include the semantic features of the traffic such as the application layer protocol, keywords, regular patterns, etc. The deep packet inspection technology can be used to perform semantic analysis on the application layer payload content of the traffic data packet to extract the semantic features of the traffic such as the application layer protocol type, keyword frequency and regular pattern matching degree.

[0101] The multi-level feature representation is input into a feature fusion module for feature fusion processing to obtain a corresponding deep feature representation; wherein, the deep feature representation can be understood as first using a feature selection algorithm such as information gain or chi-square test to calculate the importance weights of the above-mentioned features of different dimensions, and then performing weighted average fusion processing on the multi-level feature representation based on the obtained importance weights to obtain a comprehensive feature vector that can improve the ability to recognize complex attack patterns, and using the maximum pooling method to reduce the dimension of the comprehensive feature vector to extract significant features of malicious traffic.

[0102] The deep feature representation is input into the fully connected layer for classification prediction to obtain the corresponding malicious traffic identification result; wherein the malicious traffic identification result can be understood as normal traffic or malicious traffic, and the type corresponding to the malicious traffic is one of the above-mentioned malicious traffic prediction category sets.

[0103] When the malicious traffic identification result is malicious traffic, the corresponding network traffic is marked as a suspected zero-day attack sample; in actual applications, in order to ensure the timeliness of abnormal traffic prevention and control, this embodiment preferably reports abnormal alarms while identifying suspected zero-day attack samples, and generates abnormal alarm log information to provide a traceability basis for subsequent traffic detection results analysis.

[0104] In order to ensure the accuracy of abnormal traffic prevention and control, this embodiment preferably performs attack type analysis on suspected zero-day attack samples based on the above analysis to determine the existence of zero-day attack risks, so as to perceive the changes in malicious traffic attack patterns and attack methods in real time and adaptively optimize traffic management strategies. Specifically, the steps of identifying the attack type based on the suspected zero-day attack samples and obtaining the corresponding zero-day attack type include:

[0105] Obtain a key feature vector corresponding to the suspected zero-day attack sample; the key feature vector includes packet flow characteristics, transmission frequency characteristics and communication transmission characteristics; wherein the packet flow characteristics include average packet flow, maximum packet flow and packet flow standard deviation, the transmission frequency characteristics include transmission frequency mean, transmission frequency peak and transmission frequency change trend, and the communication transmission characteristics include entropy value, isomorphism and regional distribution of source and target IP addresses; it should be noted that each characteristic value in the above-mentioned packet flow characteristics, transmission frequency characteristics and communication transmission characteristics can be obtained by referring to relevant existing technologies and will not be described in detail here.

[0106] The similarity between the key feature vector and the feature vectors of each known attack type in the preset zero-day attack feature library is calculated by the cosine similarity algorithm, and it is determined whether the average similarity is greater than the preset similarity threshold; wherein, the preset zero-day attack feature library can be understood as a database of key feature vectors corresponding to all known attack types constructed based on historical experience or expert knowledge, and the database can be adaptively adjusted as the type of network attack changes; the corresponding preset similarity threshold can be set according to actual application requirements, and is not specifically limited here.

[0107] If so, the known attack type corresponding to the maximum similarity is used as the zero-day attack type of the suspected zero-day attack sample; otherwise, the suspected zero-day attack sample is determined to be a new type of zero-day attack, and the key feature vector of the suspected zero-day attack sample is added to the preset zero-day attack feature library. In practical applications, when the average value of the similarity between the key feature vector obtained by the cosine similarity algorithm and all the feature vectors of the known attack types in the preset zero-day attack feature library is higher than the preset similarity threshold, the currently identified zero-day attack sample is determined to be a known attack type, and the known attack type with the largest similarity value can be directly selected as the zero-day attack type corresponding to the currently identified zero-day attack sample; when the actual calculated average similarity is less than or equal to the preset similarity threshold, it is determined that the currently identified zero-day attack sample does not belong to a known attack type, but to a new type of zero-day attack, and its corresponding key feature vector can be added to the preset zero-day attack feature library for subsequent detection and defense, so that it can adapt to the newly emerging zero-day attacks and effectively improve the real-time detection and defense effects of zero-day attacks.

[0108] This embodiment not only realizes comprehensive and in-depth analysis and identification of malicious traffic by using multi-dimensional features including time, space and semantics in combination with a recursive neural network model, thereby improving the reliability and efficiency of zero-day attack identification, but also identifies and perceives new attack types by performing similarity analysis between identified zero-day attack samples and known attack types in a preset zero-day attack feature library. It can adaptively adjust zero-day attack detection, timely discover and respond to zero-day attacks, and provide reliable protection for real-time and effective defense against zero-day attacks.

[0109] S15. According to the zero-day attack type, obtain the corresponding malicious traffic prevention and control strategy, and conduct a comprehensive analysis of the malicious traffic prevention and control strategy and the initial traffic management strategy to generate an optimal traffic management strategy; wherein the malicious traffic prevention and control strategy can be understood as a traffic control strategy corresponding to the zero-day attack type. Taking into account that in actual applications, the zero-day attack type obtained by real-time analysis may be a known attack type or a new attack type, and the traffic prevention and control strategies applicable to different attack types are also necessarily different, this embodiment preferably obtains the corresponding effective prevention and control strategy according to the determined zero-day attack type, so as to improve the accuracy and reliability of real-time prevention and control. Specifically, the step of obtaining the corresponding malicious traffic prevention and control strategy according to the zero-day attack type includes:

[0110] If the zero-day attack type is a known attack type, then the traffic security management policy corresponding to the known attack type is obtained from the preset attack pattern knowledge base, and the traffic security management policy is used as the malicious traffic prevention and control policy; wherein, the preset attack pattern knowledge base can be understood as a database including the optimal prevention and control policies corresponding to various known attack types, and the database can also be constructed based on historical experience or expert analysis, which will not be described in detail here. In practical applications, in order to provide high efficiency and timeliness of traffic control, this embodiment preferably, when the zero-day attack type of the zero-day attack sample currently detected belongs to one of the known attack types, directly search the preset attack type traffic security management policy corresponding to the known attack type from the preset attack pattern knowledge base as the malicious traffic prevention and control policy currently required to be used, such as real-time blocking of the known attack type, etc.

[0111] If the zero-day attack type is a new type of zero-day attack, the basic traffic characteristics of the suspected zero-day attack sample are evaluated for abnormalities according to the preset isolation forest model, a corresponding attack analysis report is generated, and traffic prevention and control predictions are performed according to the attack analysis report and the preset reinforcement learning model to obtain the malicious traffic prevention and control strategy; wherein, the basic traffic characteristics include traffic surge magnitude, access frequency and data packet size distribution; in actual applications, if the zero-day attack type is a new type of zero-day attack, the basic traffic characteristics can be evaluated for abnormalities according to the preset isolation forest model first, and then a corresponding attack analysis report including information such as attack type, source IP address and destination port is generated according to the obtained abnormal score value, and then the TF-IDF algorithm is used to extract keywords from the attack analysis report, and the words with the highest weights before the preset number are selected as the state variables of the preset reinforcement learning model input, and the preset reinforcement learning model performs next action prediction analysis according to the input state variables to obtain the required malicious traffic prevention and control strategy. It should be noted that the preset reinforcement learning model in this embodiment can be constructed by training with relevant historical data, and in actual applications, it can be adaptively optimized and adjusted periodically according to the collected network traffic data, so as to continuously improve the adaptive optimization capability, prediction accuracy and timeliness of the traffic prevention and control strategy corresponding to the new attack type output based on the reinforcement learning model, and provide solid protection for malicious traffic prevention and control in complex network environments.

[0112] The optimal traffic management strategy in this embodiment can be understood as a comprehensive analysis of the initial traffic management strategy obtained based on outlier detection and the targeted prevention and control strategy for malicious traffic attack types obtained based on in-depth detection and analysis, and a more accurate, reliable and comprehensive malicious traffic prevention and control strategy applicable to the current malicious traffic attack type is obtained by directly retaining the same items in the two strategies and retaining the more comprehensive difference items in the two. It should be noted that the specific content of the optimal traffic management strategy varies depending on the type of zero-day attack obtained by real-time detection in the actual application scenario, and is not specifically limited here.

[0113] The present embodiment provides real-time acquisition of network traffic data, obtains suspected malicious traffic slice data of network traffic data according to a pre-built network traffic feature library, performs attack mode correlation analysis based on the outlier detection results of outlier detection on the suspected malicious traffic slice data to obtain an abnormal threat level, and according to the abnormal threat level, automatically generates a corresponding initial traffic management strategy based on a preset inference engine, performs noise reduction processing on the suspected malicious traffic slice data to obtain corresponding noise reduction slice data, and inputs the multi-dimensional feature data obtained by performing multi-dimensional feature extraction on the noise reduction slice data based on a preset feature engineering technology into a pre-built malicious traffic detection model to perform malicious traffic identification to obtain suspected zero-day attack samples and conduct an attack The corresponding zero-day attack type is obtained by type identification, and the corresponding malicious traffic prevention and control strategy is obtained according to the zero-day attack type. The initial traffic management strategy is adjusted according to the malicious traffic prevention and control strategy to generate a technical solution for the optimal traffic management strategy. This can effectively solve the problem that the existing network malicious traffic detection method cannot adapt to the ever-changing attack means and patterns, which makes it easy to generate a high number of false alarms and missed alarms in a complex network environment where malicious traffic presents the characteristics of diversity, concealment and deception. It can adaptively perceive the changes in traffic patterns and attack means in a complex network environment, deeply explore the hidden attack modes of malicious traffic, and quickly and accurately identify and block zero-day attacks, providing reliable protection for network security defense.

[0114] In addition, considering that there are cases where the malicious traffic detection model may make mistakes in identification in actual applications, in order to ensure the continuous reliability and stability of real-time malicious traffic detection and reduce the probability of prevention and control errors as much as possible, this embodiment preferably analyzes the false positives and false negatives of the malicious traffic detection model in actual applications and performs feedback optimization and adjustment on the malicious traffic detection model based on the analysis results. Specifically, Figure 2 As shown, the method also includes:

[0115] S16. Continuously monitor the recognition error traffic data of the malicious traffic detection model, classify the recognition error patterns according to the recognition error traffic data, and adjust and update the malicious traffic detection model according to the corresponding recognition error patterns; wherein the recognition error traffic data includes malicious false alarm traffic data and malicious missed alarm traffic data, both of which can be obtained by comprehensively analyzing the alarm information reported by the zero-day attack and the relevant log information of the network traffic access processing, and are stored in the preset recognition error database (MySQL) according to the field structure such as timestamp, source IP, destination IP and detection result to form structured data records for subsequent analysis.

[0116] In practical applications, the continuously collected recognition error traffic data is cleaned, standardized, and denoised before being converted into a high-quality data format suitable for analysis. The data analysis engine is then used to deeply mine the preprocessed false positive and false negative data, and data mining algorithms such as clustering and association analysis are used to discover the hidden patterns and laws in the false positive and false negative data, and to find out the key factors that lead to false positive and false negative. For example, the Spark MLlib machine learning library is used to extract features from the preprocessed recognition error traffic data, and 10 key feature data such as source IP, destination IP, port number, protocol type, packet length, and traffic rate are selected. The K-means clustering algorithm is used to classify the key feature data into recognition error patterns, and the clustering effect is evaluated by the Silhouette coefficient. When the coefficient is greater than 8, the corresponding clustering result is considered acceptable, and the center points of each cluster extracted at this time are taken as typical false positive and false negative patterns, and the common features of various recognition error patterns are obtained. Then, the association rule mining algorithm is used to analyze the correlation between the common features of various recognition error patterns and the factors affecting the detection effect of the malicious traffic detection model, and the key factors that lead to recognition errors are obtained. Model influencing factors or key model influencing factor combinations; finally, formulate effective optimization strategies based on the obtained key model influencing factors or key model influencing factor combinations (such as dynamically adjusting the feature fusion weights in the model, changing the sensitivity of the detection model to different features, etc.), and use machine learning methods such as incremental learning or parameter fine-tuning to optimize and adjust the relevant model parameters and training methods in the current malicious traffic detection model. At the same time, the L1 regularization method is used to prevent overfitting and improve the generalization ability of the model, so that the model can be continuously optimized and upgraded, and the model prediction performance can be continuously improved. While timely discovering and blocking real malicious traffic, the error probability of subsequent malicious traffic identification is reduced, the accuracy and robustness of detection are improved, and the reliability of real-time prevention and control of malicious traffic is thereby improved.

[0117] The embodiment of the present application obtains network traffic data in real time, obtains suspected malicious traffic slice data of the network traffic data according to a pre-built network traffic feature library, performs attack pattern correlation analysis on the outlier detection results of the suspected malicious traffic slice data to obtain the abnormal threat level, and automatically generates the corresponding initial traffic management strategy based on the preset inference engine according to the abnormal threat level, performs noise reduction processing on the suspected malicious traffic slice data to obtain the corresponding noise reduction slice data, performs multi-dimensional feature extraction on the noise reduction slice data based on the preset feature engineering technology to obtain the multi-dimensional feature data, inputs the pre-built malicious traffic detection model to perform malicious traffic identification to obtain suspected zero-day attack samples, and performs attack type identification to obtain the corresponding zero-day attack type, and obtains the corresponding zero-day attack type according to the zero-day attack type. Malicious traffic prevention and control strategy, and adjust the initial traffic management strategy according to the malicious traffic prevention and control strategy to generate the optimal traffic management strategy, and continuously monitor the recognition error traffic data of the malicious traffic detection model, and classify the recognition error patterns according to the recognition error traffic data, and adjust and update the malicious traffic detection model according to the corresponding recognition error patterns. The scheme can not only adaptively perceive the changes in traffic patterns and attack methods in complex network environments, deeply explore the hidden attack patterns of malicious traffic, and quickly and accurately identify and block zero-day attacks, but also continuously iterate and optimize the malicious traffic detection model through the established recognition error analysis feedback mechanism, so that the malicious traffic detection capability is continuously improved, thereby realizing continuous, stable, intelligent and adaptive network security defense.

[0118] It should be noted that although the steps in the above flowchart are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders.

[0119] In one embodiment, Figure 3 As shown, a network malicious traffic detection and analysis system is provided, the system comprising:

[0120] The traffic slice processing module 1 is used to obtain network traffic data and obtain suspected malicious traffic slice data of the network traffic data according to a pre-built network traffic feature library;

[0121] Initial strategy generation module 2, used to perform outlier detection on the suspected malicious traffic slice data, perform attack mode association analysis according to the corresponding outlier detection results, obtain the abnormal threat level, and automatically generate the corresponding initial traffic management strategy based on the preset reasoning engine according to the abnormal threat level;

[0122] The multi-dimensional feature extraction module 3 is used to perform noise reduction processing on the suspected malicious traffic slice data to obtain corresponding noise reduction slice data, and perform multi-dimensional feature extraction on the noise reduction slice data based on a preset feature engineering technology to obtain corresponding multi-dimensional feature data; the multi-dimensional feature data includes time dimension feature data, space dimension feature data and semantic space feature data;

[0123] A zero-day attack identification module 4 is used to input the multi-dimensional feature data into a pre-built malicious traffic detection model to identify malicious traffic, obtain suspected zero-day attack samples, and identify attack types based on the suspected zero-day attack samples to obtain corresponding zero-day attack types;

[0124] The optimal strategy generation module 5 is used to obtain the corresponding malicious traffic prevention and control strategy according to the zero-day attack type, and comprehensively analyze the malicious traffic prevention and control strategy and the initial traffic management strategy to generate the optimal traffic management strategy.

[0125] In one embodiment, Figure 4 As shown, a network malicious traffic detection and analysis system is provided, the system comprising:

[0126] The model continuous updating module 6 is used to continuously monitor the recognition error traffic data of the malicious traffic detection model, classify the recognition error patterns according to the recognition error traffic data, and adjust and update the malicious traffic detection model according to the corresponding recognition error patterns; the recognition error traffic data includes malicious false alarm traffic data and malicious missed alarm traffic data.

[0127] For the specific limitations of the network malicious traffic detection and analysis system, please refer to the limitations of the network malicious traffic detection and analysis method above, and the corresponding technical effects can also be obtained equivalently, which will not be repeated here. Each module in the above-mentioned network malicious traffic detection and analysis system can be implemented in whole or in part through software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0128] In summary, the embodiment of the present invention provides a network malicious traffic detection and analysis method and system, wherein the network malicious traffic detection and analysis method realizes real-time acquisition of network traffic data, obtains suspected malicious traffic slice data of network traffic data according to a pre-constructed network traffic feature library, performs attack mode association analysis on the outlier detection result of outlier detection on the suspected malicious traffic slice data to obtain an abnormal threat level, and according to the abnormal threat level, automatically generates a corresponding initial traffic management strategy based on a preset inference engine, performs noise reduction processing on the suspected malicious traffic slice data to obtain corresponding noise reduction slice data, and inputs the multi-dimensional feature data obtained by performing multi-dimensional feature extraction on the noise reduction slice data based on a preset feature engineering technology into a pre-constructed malicious traffic detection model to perform malicious traffic identification to obtain suspected zero-day attack samples and perform attack type identification to obtain corresponding zero-day attack samples. The invention discloses a technical scheme for determining the attack type, obtaining the corresponding malicious traffic prevention and control strategy according to the zero-day attack type, adjusting the initial traffic management strategy according to the malicious traffic prevention and control strategy, generating the optimal traffic management strategy, continuously monitoring the recognition error traffic data of the malicious traffic detection model, classifying the recognition error pattern according to the recognition error traffic data, and adjusting and updating the malicious traffic detection model according to the corresponding recognition error pattern. The method can not only adaptively perceive the changes in attack means and attack patterns in a complex network environment, deeply mine the hidden attack patterns of malicious traffic, and realize rapid and accurate identification and blocking of zero-day attacks, but also continuously iterate and optimize the malicious traffic detection model through the established recognition error analysis feedback mechanism, so that the malicious traffic detection capability is continuously improved, thereby realizing continuous, stable, intelligent and adaptive network security defense.

[0129] Each embodiment in this specification is described in a progressive manner, and the same or similar parts of each embodiment can be directly referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. It should be noted that the technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above-mentioned embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0130] The above-mentioned embodiments only express several preferred implementation modes of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in the technical field, several improvements and substitutions can be made without departing from the technical principles of the present invention, and these improvements and substitutions should also be regarded as the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be based on the protection scope of the claims.

Claims

1. A network malicious traffic detection and analysis method, characterized in that: The method comprises the following steps: Acquire network traffic data in real time, and acquire suspected malicious traffic slice data of the network traffic data according to a pre-built network traffic feature library; wherein the suspected malicious traffic slice data is obtained based on executing a preset malicious traffic slice operation on the suspected malicious traffic in the network traffic data; the preset malicious traffic slice operation is to extract data feature information from the original traffic data according to a slice traffic range determined according to the analysis result of the suspected malicious traffic; Perform outlier detection on the suspected malicious traffic slice data, perform attack mode association analysis based on the corresponding outlier detection results, obtain the abnormal threat level, and automatically generate a corresponding initial traffic management strategy based on a preset reasoning engine according to the abnormal threat level; The suspected malicious traffic slice data is subjected to noise reduction processing to obtain corresponding noise reduction slice data, and multi-dimensional feature extraction is performed on the noise reduction slice data based on a preset feature engineering technology to obtain corresponding multi-dimensional feature data; the multi-dimensional feature data includes time dimension feature data, space dimension feature data and semantic space feature data; the time dimension feature data includes the duration, periodicity and burstiness of the traffic; the space dimension feature data includes the geographical distribution, topological structure and node relationship of the traffic, the geographical distribution is obtained by clustering analysis of the source and destination IP addresses, the topological structure and node relationship are obtained by first calculating the distance index between IP addresses to construct an IP address association graph, and then analyzing based on the IP address association graph; the semantic space feature is obtained by using deep packet inspection technology to perform semantic analysis on the application layer payload content of the traffic data packet, including the application layer protocol type of the traffic, keyword frequency and regular pattern matching degree; The multi-dimensional feature data is input into a pre-built malicious traffic detection model to identify malicious traffic, obtain suspected zero-day attack samples, and identify the attack type based on the suspected zero-day attack samples to obtain the corresponding zero-day attack type; the malicious traffic detection model is obtained based on recursive neural network training; the recursive neural network includes an input layer, a multi-dimensional feature extraction module, a feature fusion module and a fully connected layer; the multi-dimensional feature extraction module includes a bidirectional long short-term memory network; the malicious traffic prediction category set used in the malicious traffic detection model training is generated based on the typical malicious traffic pattern obtained by clustering analysis of the historical multi-dimensional feature data set corresponding to the historical noise reduction slice data set; According to the zero-day attack type, a corresponding malicious traffic prevention and control strategy is obtained, and the malicious traffic prevention and control strategy and the initial traffic management strategy are comprehensively analyzed to generate an optimal traffic management strategy; The step of identifying the attack type according to the suspected zero-day attack sample to obtain the corresponding zero-day attack type includes: Obtaining a key feature vector corresponding to the suspected zero-day attack sample; the key feature vector includes a data packet flow feature, a transmission frequency feature, and a communication transmission feature; Calculate the similarity between the key feature vector and each known attack type feature vector in the preset zero-day attack feature library by using the cosine similarity algorithm, and determine whether the average similarity is greater than a preset similarity threshold; If yes, the known attack type corresponding to the maximum similarity is used as the zero-day attack type of the suspected zero-day attack sample; otherwise, the suspected zero-day attack sample is determined to be a new type of zero-day attack, and the key feature vector of the suspected zero-day attack sample is added to the preset zero-day attack feature library; The step of obtaining a corresponding malicious traffic prevention and control strategy according to the zero-day attack type includes: If the zero-day attack type is a new type of zero-day attack, an abnormal assessment is performed on the basic traffic characteristics of the suspected zero-day attack sample according to the preset isolation forest model, a corresponding attack analysis report is generated, and traffic prevention and control prediction is performed according to the attack analysis report and the preset reinforcement learning model to obtain the malicious traffic prevention and control strategy; the basic traffic characteristics include traffic surge magnitude, access frequency and data packet size distribution; wherein, the step of performing traffic prevention and control prediction according to the attack analysis report and the preset reinforcement learning model to obtain the malicious traffic prevention and control strategy includes: using the TF-IDF algorithm to extract keywords from the attack analysis report, using the first preset number of words with the highest weight as the state variables input into the preset reinforcement learning model to perform next action prediction analysis, and obtaining the malicious traffic prevention and control strategy; The method further comprises: Continuously monitor the recognition error traffic data of the malicious traffic detection model, classify recognition error patterns according to the recognition error traffic data, use association rule mining algorithm to analyze the correlation between common features of various recognition error patterns and factors affecting the detection effect of the malicious traffic detection model, and formulate an optimization strategy based on the obtained key model influencing factors or a combination of key model influencing factors, and use incremental learning or parameter fine-tuning to optimize the model parameters and training methods corresponding to the optimization strategy, so as to adjust and update the malicious traffic detection model; the recognition error traffic data includes malicious false alarm traffic data and malicious missed alarm traffic data.

2. The network malicious traffic detection and analysis method according to claim 1, characterized in that: The suspected malicious traffic slice data includes the source IP address, destination port, payload content, payload size and transmission frequency of the data packet; The step of obtaining suspected malicious traffic slice data of the network traffic data according to the pre-built network traffic feature library includes: According to the network traffic feature library, extracting features from the network traffic data based on a statistical analysis method to obtain initial feature data; According to a preset attention mechanism, the fusion weight of each feature data in the initial feature data is calculated based on the malicious traffic correlation, and the initial feature data is weightedly fused according to each fusion weight to obtain a corresponding high-dimensional feature vector; The high-dimensional feature vector is subjected to dimensionality reduction processing and then input into a pre-built long short-term memory neural network for traffic classification to obtain a corresponding traffic classification result; When the traffic classification result is suspected malicious traffic, a preset malicious traffic slicing operation is performed to extract corresponding suspected malicious traffic slicing data.

3. The network malicious traffic detection and analysis method according to claim 1, characterized in that: The step of performing outlier detection on the suspected malicious traffic slice data, performing attack mode association analysis according to the corresponding outlier detection result, and obtaining the abnormal threat level includes: According to the preset outlier detection model, an outlier detection is performed on each slice feature data in the suspected malicious traffic slice data to obtain a corresponding outlier detection result; Cluster analysis is performed on the slice feature data with all abnormal detection results marked as outliers to obtain multiple abnormal traffic patterns; Based on the association rule mining technology, various abnormal traffic patterns are analyzed for correlation with known attack patterns to obtain corresponding correlation analysis results; According to the correlation analysis result and the preset abnormal threat knowledge base, the corresponding abnormal threat level is obtained, and based on the online incremental learning mechanism, the preset abnormal value detection model and the preset abnormal threat knowledge base are continuously updated.

4. The network malicious traffic detection and analysis method according to claim 1, characterized in that: The step of performing noise reduction processing on the suspected malicious traffic slice data to obtain corresponding noise reduction slice data comprises: Performing noise type identification on the suspected malicious traffic slice data to obtain a corresponding noise type; If the noise type is periodic stationary noise, the suspected malicious traffic slice data is subjected to frequency domain filtering processing by a preset bandpass filter to obtain the noise-reduced slice data; otherwise, a corresponding wavelet transform model is constructed according to a preset wavelet basis function and a decomposition layer number; According to the wavelet transform model, multi-scale wavelet decomposition is performed on the suspected malicious traffic slice data to obtain corresponding wavelet coefficients; According to a preset high-frequency threshold, the high-frequency components in the wavelet coefficients are identified, and the high-frequency components are set to zero to obtain corresponding denoised wavelet coefficients; Using a wavelet reconstruction algorithm, the noise reduction wavelet coefficients are reconstructed into a time domain signal to obtain corresponding noise reduction feature data; Calculating a signal-to-noise ratio of the noise reduction feature data, and obtaining a corresponding noise reduction performance evaluation result according to the signal-to-noise ratio; When the noise reduction performance evaluation result reaches the preset noise reduction performance standard, the noise reduction feature data is used as the noise reduction slice data; When the noise reduction performance evaluation result does not reach the preset noise reduction performance standard, the wavelet transform model and the preset high-frequency threshold are optimized, and the noise reduction feature data is subjected to noise reduction processing according to the optimized wavelet transform model and the preset high-frequency threshold until the preset noise reduction performance standard is reached, thereby obtaining the noise reduction slice data.

5. The network malicious traffic detection and analysis method according to claim 1, characterized in that: The training steps of the malicious traffic detection model include: Acquire a historical network traffic data set, and process the historical network traffic data set to obtain a corresponding historical noise reduction slice data set; According to the preset feature engineering technology, multi-dimensional feature extraction is performed on the historical denoised slice data set to obtain a corresponding historical multi-dimensional feature data set; Performing cluster analysis on the historical multi-dimensional feature data set to obtain a plurality of typical malicious traffic patterns, and obtaining a malicious traffic prediction category set based on the typical malicious traffic patterns; According to the historical multi-dimensional feature data set and the malicious traffic prediction category set, performing parameter optimization training on the recursive neural network to obtain an initial traffic detection model; According to the knowledge distillation processing technology, the initial traffic detection model is compressed to obtain the malicious traffic detection model.

6. The network malicious traffic detection and analysis method according to claim 5, characterized in that: The step of inputting the multi-dimensional feature data into a pre-built malicious traffic detection model to identify malicious traffic and obtain suspected zero-day attack samples includes: Inputting the multi-dimensional feature data into the multi-dimensional feature extraction module for feature extraction to obtain a multi-level feature representation; the multi-level feature representation includes time dimension features, space dimension features and semantic space features; Inputting the multi-level feature representation into a feature fusion module for feature fusion processing to obtain a corresponding deep feature representation; Inputting the deep feature representation into the fully connected layer for classification prediction to obtain corresponding malicious traffic identification results; When the malicious traffic identification result is malicious traffic, the corresponding network traffic is marked as a suspected zero-day attack sample.

7. The network malicious traffic detection and analysis method according to claim 1, characterized in that: The step of obtaining a corresponding malicious traffic prevention and control strategy according to the zero-day attack type includes: If the zero-day attack type is a known attack type, a traffic security management policy corresponding to the known attack type is obtained from a preset attack pattern knowledge base, and the traffic security management policy is used as the malicious traffic prevention and control policy.

8. A network malicious traffic detection and analysis system, characterized in that: The system comprises: A traffic slicing processing module is used to obtain network traffic data and obtain suspected malicious traffic slice data of the network traffic data according to a pre-built network traffic feature library; wherein the suspected malicious traffic slice data is obtained based on performing a preset malicious traffic slicing operation on the suspected malicious traffic in the network traffic data, and the preset malicious traffic slicing operation is to extract data feature information from the original traffic data according to a slice traffic range determined according to the analysis result of the suspected malicious traffic; An initial strategy generation module is used to perform outlier detection on the suspected malicious traffic slice data, perform attack mode association analysis based on the corresponding outlier detection results, obtain the abnormal threat level, and automatically generate a corresponding initial traffic management strategy based on a preset reasoning engine according to the abnormal threat level; A multi-dimensional feature extraction module is used to perform noise reduction processing on the suspected malicious traffic slice data to obtain corresponding noise reduction slice data, and perform multi-dimensional feature extraction on the noise reduction slice data based on a preset feature engineering technology to obtain corresponding multi-dimensional feature data; the multi-dimensional feature data includes time dimension feature data, space dimension feature data and semantic space feature data; the time dimension feature data includes the duration, periodicity and burstiness of the traffic; the space dimension feature data includes the geographical distribution, topological structure and node relationship of the traffic, the geographical distribution is obtained by clustering the source and destination IP addresses, the topological structure and node relationship are obtained by first calculating the distance index between the IP addresses to construct an IP address association graph, and then analyzing it based on the IP address association graph; the semantic space feature is obtained by performing semantic analysis on the application layer load content of the traffic data packet using deep packet inspection technology, including the application layer protocol type of the traffic, keyword frequency and regular pattern matching degree; A zero-day attack identification module is used to input the multi-dimensional feature data into a pre-built malicious traffic detection model to identify malicious traffic, obtain suspected zero-day attack samples, and identify attack types based on the suspected zero-day attack samples to obtain corresponding zero-day attack types; An optimal strategy generation module is used to obtain a corresponding malicious traffic prevention and control strategy according to the zero-day attack type, and to comprehensively analyze the malicious traffic prevention and control strategy and the initial traffic management strategy to generate an optimal traffic management strategy; The step of identifying the attack type according to the suspected zero-day attack sample to obtain the corresponding zero-day attack type includes: Obtaining a key feature vector corresponding to the suspected zero-day attack sample; the key feature vector includes a data packet flow feature, a transmission frequency feature, and a communication transmission feature; Calculate the similarity between the key feature vector and each known attack type feature vector in the preset zero-day attack feature library by using the cosine similarity algorithm, and determine whether the average similarity is greater than a preset similarity threshold; If yes, the known attack type corresponding to the maximum similarity is used as the zero-day attack type of the suspected zero-day attack sample; otherwise, the suspected zero-day attack sample is determined to be a new type of zero-day attack, and the key feature vector of the suspected zero-day attack sample is added to the preset zero-day attack feature library; The obtaining of a corresponding malicious traffic prevention and control strategy according to the zero-day attack type includes: If the zero-day attack type is a new type of zero-day attack, an abnormal assessment is performed on the basic traffic characteristics of the suspected zero-day attack sample according to the preset isolation forest model, a corresponding attack analysis report is generated, and traffic prevention and control prediction is performed according to the attack analysis report and the preset reinforcement learning model to obtain the malicious traffic prevention and control strategy; the basic traffic characteristics include traffic surge magnitude, access frequency and data packet size distribution; wherein, the step of performing traffic prevention and control prediction according to the attack analysis report and the preset reinforcement learning model to obtain the malicious traffic prevention and control strategy includes: using the TF-IDF algorithm to extract keywords from the attack analysis report, using the first preset number of words with the highest weight as the state variables input into the preset reinforcement learning model to perform next action prediction analysis, and obtaining the malicious traffic prevention and control strategy; The system further comprises: A model continuous updating module is used to continuously monitor the recognition error traffic data of the malicious traffic detection model, and classify recognition error patterns according to the recognition error traffic data, use an association rule mining algorithm to analyze the correlation between the common characteristics of various recognition error patterns and the detection effect influencing factors of the malicious traffic detection model, and formulate an optimization strategy based on the obtained key model influencing factors or a combination of key model influencing factors and use incremental learning or parameter fine-tuning to optimize the model parameters and training methods corresponding to the optimization strategy, so as to adjust and update the malicious traffic detection model; the recognition error traffic data includes malicious false alarm traffic data and malicious missed alarm traffic data.

Images