Abnormal network traffic analysis method and system based on deep learning

Abstract

The present invention discloses a method and system for analyzing abnormal network traffic based on deep learning, including: collecting network traffic data and preprocessing it, extracting features from the preprocessed network traffic data; constructing a model adaptive adjustment framework for detecting abnormal network traffic, using the extracted features to train the model adaptive adjustment framework, and using the trained model adaptive adjustment framework to detect abnormal network traffic; for the detected abnormal traffic, using the abnormal correlation analysis technology in the network traffic to analyze the correlation between abnormal events and potential attack patterns; forming a feedback data set according to the analysis results, and optimizing and adjusting the model adaptive adjustment framework according to the feedback data set; generating a security report and corresponding response suggestions based on the analysis results of the aforementioned steps. The present invention can accurately extract and analyze complex features in network traffic, and effectively improve the accuracy and sensitivity of abnormal traffic detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer network security, and specifically relates to an abnormal network traffic analysis method and system based on deep learning. Background Art

[0002] In today's digital age, network security has become the cornerstone of Internet technology and services. As network attacks become more complex and covert, traditional rule- and signature-based intrusion detection systems (IDS) are gradually becoming incapable of coping with them. These systems rely on the characteristics of known attacks and are unable to effectively respond to zero-day attacks and advanced persistent threats (APTs). Therefore, methods for automatic analysis of abnormal network traffic using deep learning have attracted widespread attention, aiming to identify potential abnormal behaviors by learning the normal patterns of network traffic.

[0003] Deep learning, as an algorithm that can automatically extract high-level features from large amounts of data, has demonstrated its powerful performance in many fields, including image recognition, natural language processing, and complex game strategies. In recent years, researchers have tried to apply deep learning technology to the field of network security, especially in abnormal traffic detection. Although these methods have shown potential, they still face several challenges in practical applications.

[0004] The complexity and dynamics of the network environment require that the model should not only be able to accurately identify known abnormal patterns, but also be able to adapt to emerging attack methods. However, most deep learning models are static after training and lack the ability to self-adjust based on new data. At the same time, the diversity and high dimensionality of network traffic data require the model to be able to extract and utilize complex features, which places higher requirements on the design of the model. Traditional deep learning models often operate like black boxes and lack interpretability of their decision-making process, which is unacceptable in the field of network security because security analysts need to understand the reasons behind the detected anomalies. Therefore, it is necessary to develop an abnormal network traffic analysis method that can overcome the limitations of existing methods. Summary of the invention

[0005] One purpose of the present invention is to address the deficiencies in the prior art and provide a method for analyzing abnormal network traffic based on deep learning, which can accurately extract and analyze complex features in network traffic and effectively improve the accuracy and sensitivity of abnormal traffic detection.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] A method for analyzing abnormal network traffic based on deep learning, comprising the following steps:

[0008] Step 1: collect and preprocess network traffic data, and use a deep learning model to extract features from the preprocessed network traffic data;

[0009] Step 2, constructing a model adaptive adjustment framework for detecting abnormal network traffic, using the features extracted in step 1 to train the model adaptive adjustment framework, and using the trained model adaptive adjustment framework to detect abnormal network traffic;

[0010] Step 3: For the detected abnormal traffic, the abnormal correlation analysis technology in network traffic is used to analyze the correlation between abnormal events and potential attack patterns;

[0011] Step 4: forming a feedback data set according to the analysis results of step 3, and optimizing and adjusting the model adaptability adjustment framework according to the feedback data set to obtain an optimized model adaptability adjustment framework;

[0012] Step 5: Based on the analysis results of the previous steps, the model adaptive adjustment framework generates a safety report and corresponding response recommendations.

[0013] Furthermore, the specific implementation of step 1 includes:

[0014] Step 1.1: Collect raw network traffic data from network devices and security systems, remove incomplete or damaged data records, and filter out data fields related to security analysis;

[0015] Step 1.2: Convert network traffic data from different sources and formats into a unified format;

[0016] Step 1.3, normalizing the network traffic data to obtain normalized network traffic data;

[0017] Step 1.4 improves the ResNet-50 model and uses the improved ResNet-50 model to extract key features from the normalized network traffic data in step 1.3;

[0018] Step 1.5 Use the improved ResNet-50 for feature selection.

[0019] Furthermore, the improved ResNet-50 network structure is:

[0020] The improved ResNet-50 network structure includes:

[0021] Convolutional layer, used to extract features;

[0022] Pooling layer, used to reduce the spatial size of the feature map;

[0023] Convolution blocks and identity blocks are stacked layers, where each convolution block includes three convolution layers and each identity block includes four convolution layers. The convolution block is used to further extract features, while the identity block is used to maintain the size and number of channels of the feature map unchanged.

[0024] Furthermore, the format unification method in step 1.2 includes:

[0025] Unify timestamp format: convert all timestamps to ISO 8601 standard format;

[0026] Unified IP address representation: Unify IP addresses into dotted decimal format;

[0027] Data type conversion: convert the IP address field to a string type and the port number field to an integer type;

[0028] Standardize the numerical data, including: Flow size standardization: calculate the mean and standard deviation of the flow size, and use the z-score method for standardization;

[0029] Packet interval time standardization: Calculate the mean and standard deviation of the packet interval time and use the z-score method for standardization;

[0030] Protocol type encoding: Use one-hot encoding to convert the protocol type into an encoding vector.

[0031] Furthermore, the model adaptation adjustment framework includes convolutional neural network (CNN), long short-term memory network (LSTM) and a combination of the two.

[0032] Furthermore, the specific implementation of step 3 is as follows:

[0033] Step 3.1 Based on the detected abnormal network traffic events, a graph database or graph processing tool is used to construct a graph according to event features and predefined association rules, where nodes represent individual abnormal events and edges represent potential associations between events;

[0034] Step 3.2: Use the graph algorithm to calculate the shortest path between any two nodes using the shortest path algorithm to identify the key attack paths and diffusion paths, and use the community discovery algorithm to identify subgroups and communities in the graph to reveal the deep connections between different attack events.

[0035] Step 3.3 Analyze the key nodes and paths in each attack chain to determine its potential impact on the network system;

[0036] Step 3.4 provides specific defense measures and response suggestions for each identified attack mode or link based on the security impact assessment results in step 3.3.

[0037] Furthermore, the specific implementation of step 4 is as follows:

[0038] Step 4.1 Summarize the results of anomaly detection and association analysis, including the detection accuracy, false positive rate, false negative rate, and the accuracy and practicality evaluation of association analysis, to form a comprehensive feedback data set;

[0039] Step 4.2 After aggregating the feedback data, evaluate the performance of the model adaptive adjustment framework based on the feedback data;

[0040] Step 4.3: According to the performance evaluation results in step 4.2, adjust the parameters and structure of the model adaptation framework;

[0041] After adjustment in step 4.4, the detection accuracy and generalization ability of the model adaptation framework are evaluated.

[0042] Furthermore, step 5 specifically includes:

[0043] Step 5.1 Summarize all key information obtained in the previous analysis steps, organize and compile the data and analysis results generated in step 4, including the identified abnormal events, attack patterns, and key affected assets, to form a comprehensive overview of the analysis results;

[0044] Step 5.2 Based on the summarized analysis results, use a report template or security information management system to automatically generate a report, ensuring that the report content includes an incident summary, analysis details, impact assessment, and targeted defense measures and response recommendations;

[0045] Step 5.3 Based on the analysis results and security impact assessment, propose specific response measures, including emergency patch application, configuration changes, isolation of affected systems, enhanced monitoring, and long-term security improvement recommendations;

[0046] Step 5.4 Distribute the security report to network administrators, security analysts, and senior management via email, security information and event management systems (SIEM), or other automated tools; and activate the emergency response team or execute pre-planned response measures based on the recommendations in the report.

[0047] Another object of the present invention is to provide a system for implementing the above-mentioned abnormal network traffic analysis method based on deep learning, comprising:

[0048] The traffic collection module is used to collect and preprocess network traffic data, and use a deep learning model to extract features from the preprocessed network traffic data;

[0049] A framework construction module is used to construct a model adaptive adjustment framework, and use the features extracted by the traffic collection module to train the model adaptive adjustment framework, and use the trained model adaptive adjustment framework to detect abnormal network traffic;

[0050] The abnormal event analysis module is used to analyze the correlation between abnormal events and potential attack patterns using abnormal correlation analysis technology in network traffic for the detected abnormal traffic;

[0051] A framework adjustment module is used to form a feedback data set according to the analysis results of the abnormal event analysis module, and optimize and adjust the model adaptability adjustment framework according to the feedback data set to obtain an optimized model adaptability adjustment framework;

[0052] The report generation module is used to generate detailed security reports and corresponding response recommendations based on the analysis results.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] 1. The present invention uses the improved ResNet-50 network technology through deep feature fusion to more accurately extract and analyze complex features in network traffic, effectively improving the accuracy and sensitivity of abnormal traffic detection, which is particularly important for identifying new and complex network attack methods;

[0055] 2. The automatic model adaptive adjustment framework introduced in this invention enables the detection model to be updated and adjusted in real time according to changes in the network environment and emerging threats. This adaptive capability ensures that the system can maintain efficient defense performance in the face of continuously evolving network threats;

[0056] 3. The present invention uses abnormal correlation analysis technology in network traffic to not only detect single abnormal events, but also analyze the correlation between abnormal events to reveal potential attack strategies and patterns. This provides security analysts with richer contextual information, which helps to quickly and accurately locate and respond to network security threats;

[0057] 4. The present invention can automatically generate a security report containing detailed analysis results and response suggestions. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 A schematic diagram of a process for automatically analyzing abnormal network traffic using deep learning according to an embodiment of the present invention;

[0059] Figure 2 Schematic diagram of abnormal correlation analysis according to an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The technical solutions in the embodiments of the present invention will be described clearly and completely below in combination with the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0062] The present invention will be further described below in conjunction with specific embodiments, but the present invention is not limited thereto.

[0063] like Figure 1 As shown, an embodiment of the present invention provides a method for automatically analyzing abnormal network traffic using deep learning, comprising the following steps:

[0064] Step 1: collect and preprocess network traffic data, and use a deep learning model to extract features from the preprocessed network traffic data;

[0065] In this step, first, the collected network traffic data is preprocessed to remove noise and reduce the complexity of data processing. Then, the ResNet-50 model improved by deep feature fusion is used to extract features from the preprocessed data. The improved ResNet-50 model uses deep learning technology to automatically identify and extract key features in network traffic data, including but not limited to traffic size, packet interval time, protocol type, etc., to provide accurate input features for subsequent anomaly detection. The specific operations are as follows:

[0066] Step 1.1 Collect raw network traffic data from network devices and security systems, remove incomplete or damaged data records, and filter out data fields related to security analysis, including: source IP address and destination IP address (used to identify the origin and destination of the data packet), source port and destination port (provide information about data transmission services), protocol type (such as TCP, UDP, ICMP, indicating the transmission protocol of the data packet), packet size (reflects the capacity of the data packet), packet interval time (helps analyze traffic patterns), and flag information (TCP flag, used to identify the status and control information of the connection);

[0067] Step 1.2 Convert network traffic data from different sources and formats into a unified format;

[0068] Among them, all timestamps are converted to ISO 8601 standard format, and IP addresses are unified into dotted decimal format (for example: 192.168.0.1);

[0069] Data type conversion: convert the IP address field to a string type, and the port number field to an integer type; perform standardization on numeric data, including: traffic size standardization and packet interval standardization, where traffic size standardization refers to calculating the mean and standard deviation of the traffic size, and using the z-score method for standardization. Packet interval standardization is to calculate the mean and standard deviation of the packet interval, and also uses the z-score method for standardization. Protocol type encoding: Use one-hot encoding to convert the protocol type into an encoding vector. Ensure that the data is at the same order of magnitude to reduce the deviation of model training;

[0070] Step 1.3: Normalize the network traffic data (such as packet size, sending frequency, and time interval) in the unified format above, and scale the data to the range of 0 to 1 using the minimum-maximum scaling method. This method is designed to accelerate the convergence speed of the model and improve the training stability.

[0071] Step 1.4 Use the ResNet-50 model improved by deep feature fusion technology to extract key features from the network traffic data processed in the above steps. Extracting key features using the improved ResNet-50 model includes the following key steps:

[0072] First, the input data passes through a convolution layer, which is used to preliminarily extract the basic features in the network traffic data, including the size and sending frequency of the data packets. These basic features provide preliminary quantitative information for the structure and behavior of the network traffic; followed by a pooling layer, the main function of which is to reduce the spatial size of the feature map to reduce the computational complexity; then, it includes four pairs of stacked convolution blocks and identity blocks, each of which consists of three convolution layers, which are used to further refine features, including pattern recognition of IP addresses and port numbers; each identity block consists of four convolution layers, which are mainly used to maintain the size and number of channels of the feature map in order to capture and retain deeper network behavior features;

[0073] Step 1.5 Use the improved ResNet-50 model to perform feature selection to enhance the feature expression ability of the network. This process is achieved through an adaptive weight adjustment mechanism that evaluates and adjusts the importance of each feature. Specifically, the improved ResNet-50 model first uses a gradient-based feature importance scoring technique to analyze the relevance of each feature in step 1.4 to the network security task. In this process, this embodiment uses the absolute value of the gradient to measure the degree of influence of each feature on the model output. A large gradient of a feature means that its change has a greater impact on the model prediction output, so it is regarded as a more important feature. This gradient-based analysis method can intuitively reflect the correlation between features and prediction tasks. The improved ResNet-50 model adjusts its weight according to the contribution of each feature to the task result. The adjustment of feature weights is implemented through a gradient descent optimization algorithm, and its optimization goal is to maximize the prediction performance of the improved ResNet-50 model. In this optimization process, the ResNet-50 model adjusts the weights of the features that contribute most to the final task prediction results, thereby further enhancing the model's ability to capture key information. This weight adjustment process is based on the feature importance score, ensuring that the focus of model optimization is on those truly important features. Finally, based on the adjusted weights, the improved ResNet-50 model selects features with weights higher than this threshold by setting a threshold, thereby reducing sensitivity to irrelevant information, enhancing the network's resistance to interference and noise, and improving the model's generalization ability. Through this process, not only is the model's performance improved in the face of complex data environments, but it also ensures the efficiency and accuracy of network security tasks.

[0074] Step 2, constructing a model adaptive adjustment framework for detecting abnormal network traffic, using the features extracted in step 1 to train the model adaptive adjustment framework, and using the trained model adaptive adjustment framework to detect abnormal network traffic;

[0075] The features extracted in step 1 are input into an automated model adaptation framework, which includes multiple deep learning models, including convolutional neural networks (CNNs), long short-term memory networks (LSTMs), and their combination (CNN-LSTM). First, the CNN layer is responsible for parsing the instantaneous features of each packet, such as packet size and protocol type, to reveal the basic structure of the packet through these structural information. Subsequently, the LSTM layer focuses on analyzing the time dependency of the packet sequence, such as the abnormal patterns of the packet sending frequency and time interval, in order to capture the dynamic changes in the time series. The CNN-LSTM combined model further integrates the above two types of information through deep feature fusion technology to optimize the representation ability of the features, that is, the spatial features extracted by the CNN layer are fused with the temporal features identified by the LSTM layer. Through a multi-layer nonlinear transformation process, not only the information expression of the features is enhanced, but also the accuracy of the model's recognition of complex data patterns is improved. Through such processing, the model can more effectively identify and respond to complex abnormal behaviors in network security, thereby significantly improving the system's protection capabilities.

[0076] After integration, the framework uses a new fusion layer to combine the analysis results of CNN and LSTM to generate a comprehensive feature representation that reflects the joint impact of instantaneous features and time series dependencies. Based on this comprehensive feature, the model adaptation framework applies a classification layer (usually a fully connected layer or a softmax layer) to calculate the anomaly probability of each network traffic sample. This probability value not only indicates whether each sample contains abnormal traffic as an output result, but also identifies and distinguishes different types of anomalies.

[0077] In order to improve the prediction ability of the model adaptive adjustment framework, it is necessary to train the model adaptive adjustment framework to adjust its parameters and structure to adapt to changes in the network environment and emerging threat types. By learning the differences between normal and abnormal traffic, the framework can accurately identify and effectively classify abnormal traffic. The specific operations are as follows:

[0078] Step 2.1 Training and Validation

[0079] Construct a data set based on the features extracted in step 1, and randomly divide the data set into a training set, a validation set, and a test set. Use the training set to train the model adaptive adjustment framework, and continuously adjust the model adaptive adjustment framework parameters through the back-propagation algorithm to minimize the loss function. Verify the performance of the model adaptive adjustment framework on the validation set, monitor overfitting, and adjust the model adaptive adjustment framework parameters or stop training early as needed to optimize the performance of the model adaptive adjustment framework. Optimize the structure and parameters of each deep learning model in the framework during the training and validation process. The framework can automatically adjust parameters according to its performance on the validation set, adjust the learning rate, change the number of network layers or nodes, etc., to achieve better detection results;

[0080] Step 2.2 Use the trained model adaptation framework to detect and classify abnormal traffic in the test set. The model adaptation framework calculates the abnormal probability of each network traffic sample and compares this probability with a pre-set threshold. If the probability exceeds the threshold, the traffic is judged to be abnormal. At the same time, the model adaptation framework can also classify the abnormalities into different types according to their characteristics, such as DoS attacks and port scans. The output of this model framework includes two parts: one is the abnormal state of each sample (normal or abnormal), and the other is the specific type of the abnormal sample.

[0081] Step 2.4 Use various performance indicators, such as accuracy, recall, and F1 score, to evaluate the detection performance of the model adaptation framework. The standard for completing the training of the model adaptation framework is set as: accuracy of at least 95%, recall of no less than 90%, and F1 score of at least 92%. Depending on the performance evaluation results, it may be necessary to return to step 2.1 to adjust and optimize the model adaptation framework again.

[0082] Step 3: For the detected abnormal traffic, the abnormal correlation analysis technology in network traffic is used to analyze the correlation between abnormal events and potential attack patterns;

[0083] For the detected abnormal traffic, we further use the abnormal correlation analysis technology in network traffic to analyze the correlation between abnormal events and potential attack patterns. This analysis is based on advanced algorithms such as graph neural networks, which can reveal the relationship between different abnormal events and identify complex attack links and potential attack sources. This step helps to understand the attacker's behavior pattern and provide important information for taking defensive measures and response strategies, such as Figure 2 As shown, the specific steps are as follows:

[0084] Step 3.1 Based on the detected abnormal network traffic events, build a graph, where nodes represent individual abnormal events and edges represent potential associations between events. Use a graph database or graph processing tool to build a graph based on event features (attack type, timestamp, affected system) and predefined association rules (events within a time window, events with the same attack target). In this graph, each node represents an independent abnormal event with attributes such as attack type, timestamp of event occurrence, and affected system. Edges between nodes represent potential associations between events. In terms of time features, use the timestamp of the event to determine the time window, identify related events that occur within a specific time range, and consider the time interval between events to capture possible attack sequences and patterns. In this way, the graph can reveal complex attack links and potential attack sources, providing more in-depth security analysis and decision support;

[0085] Step 3.2 Identify and reveal the attack chain and attack diffusion path of the abnormal events in the graph as well as the deep-level associations and patterns. This embodiment uses the shortest path algorithm to find the shortest connection path between abnormal events. The algorithm calculates the shortest distance between any two nodes in the graph, which characterizes the direct route of the attack and the possible rapid propagation direction. In this way, key attack paths and diffusion paths can be identified. At the same time, a community discovery algorithm is used to analyze the complex connections in the graph, and communities or subgroups are identified by optimizing the modularity index. This algorithm effectively reveals the densely connected areas between nodes by calculating and comparing the sum of the internal edge weights of different groups in the network with the sum of the external edge weights. The identified communities or subgroups reflect the deep-level associations between different attack events and may reveal the collective strategies or collaborative actions taken by attackers.

[0086] The key nodes and edges identified by the community discovery algorithm are the most influential and closely connected parts of the graph, and are often the core or hub of attack activities. These key nodes and edges are usually located in the center of the community or subgroup divided by the community discovery algorithm, and they play a key connecting role within the community. Because these nodes and edges have a high concentration and intermediary degree in the entire network, they are often the starting point or main propagation path of attack activities. Identifying these key nodes and edges helps to understand potential attack strategies, and through these nodes, the origin and propagation path of the attack can be traced. This information is crucial to revealing key attack steps. Through this information, we can understand the weak links or key assets that attackers may exploit, so as to strengthen protection measures and strategies in a targeted manner. By analyzing the relationship between these key nodes and edges and their communities or subgroups, security teams can more comprehensively assess the network's vulnerabilities and potential attack vectors, and implement more effective security defense and response strategies.

[0087] Step 3.3: Based on the results of the graph algorithm analysis in step 3.2, a specific security impact assessment is performed on each identified attack chain or association pattern. The key nodes and paths in each attack chain are analyzed to determine their potential impact on the network system. Focus on the affected assets, especially key servers, databases, and network devices, and assess the risks of data leakage, system paralysis, service interruption, etc. that may result from attacks on these assets. At the same time, consider the diffusion path of each attack chain and assess its potential for propagation throughout the network. A quantitative assessment is performed by setting a risk score, which assigns a risk level from low to high to each key node and path. In addition, the score also estimates the direct and indirect losses that may result from the attack, including the cost of data loss and equipment damage. Based on these risk scores, the most threatening attack chains can be clearly identified, so that targeted security protection measures can be formulated. The goal of this step is to ensure that the risk level of each key node and path accurately reflects its potential security threats, so that appropriate security measures can be taken to mitigate or avoid these risks.

[0088] Step 3.4 Based on the security impact assessment results in step 3.3, provide specific defense measures and response recommendations for each identified attack mode or link. Develop targeted defense strategies, update and strengthen intrusion detection rules for certain specific attack types, and improve the detection sensitivity of similar attacks. Then update the firewall policy to block access rights for suspicious traffic and malicious IP addresses, and enhance the security protection of the network boundary. For systems that have been affected, take isolation measures to prevent the attack from spreading further. At the same time, it is recommended to further investigate and monitor high-risk assets and key nodes to identify potential security risks and vulnerabilities. Through continuous monitoring and defense strategy updates, ensure that the network system can still maintain efficient defense capabilities in the face of new attacks, and ultimately improve overall network security.

[0089] Step 4: forming a feedback data set according to the analysis results of step 3, and optimizing and adjusting the model adaptability adjustment framework according to the feedback data set to obtain an optimized model adaptability adjustment framework;

[0090] Based on the results of anomaly detection and correlation analysis, the results are passed back to the model adaptive adjustment framework to optimize and adjust the performance of the model adaptive adjustment framework. This step ensures that the model can maintain efficient and accurate detection capabilities when facing changes in network environment and attack methods through continuous learning and adaptation. The specific steps are as follows:

[0091] Step 4.1 Summarize the results of anomaly detection and correlation analysis, including detection accuracy, false positive rate, false negative rate, and correlation analysis accuracy and practicality evaluation. This data is automatically collected through the log and reporting system, and combined with manual feedback from security analysts to form a comprehensive feedback data set;

[0092] Step 4.2 After aggregating the feedback data, input this data into the model adaptation framework to evaluate its performance. The specific evaluation process includes comprehensive testing of the model using evaluation indicators such as accuracy, recall, and F1 score. Here, accuracy measures the total correctness of the model in all detections, recall measures the model's ability to identify all actual anomalies, and F1 score is the harmonic average of accuracy and recall, which is used to evaluate the overall performance of the model. Through this evaluation, the specific weaknesses of the model adaptation framework in anomaly detection and association analysis can be identified;

[0093] Step 4.3: According to the performance evaluation results in step 4.2, find out the weaknesses of the model adaptive adjustment framework, collect data according to the weaknesses, and train the model adaptive adjustment framework to adjust its parameters and structure, so as to improve the detection accuracy and generalization ability of the model. During the training process, the hyperparameters of the model adaptive adjustment framework, such as learning rate, batch size, number of network layers, etc., are adjusted by automated machine learning (AutoML) technology to find the best parameter combination. On this basis, the data samples in the feedback data set obtained in step 4.1 are also used to fine-tune the model adaptive adjustment framework in 4.3, that is, the detected abnormal traffic samples and correlation analysis results are used as new training data to continue to train the model adaptive adjustment framework, so as to perform incremental training to ensure that the model adaptive adjustment framework can learn the latest attack patterns and features, so as to improve the detection and classification capabilities of the model adaptive adjustment framework.

[0094] Step 4.4 After completing the adjustment of the model adaptive adjustment framework, verify the performance of the adjusted model adaptive adjustment framework on an independent test set. Compare the performance of the model adaptive adjustment framework on the test set before and after the adjustment, mainly including indicators such as accuracy, recall, and F1 score, to ensure that these indicators are significantly improved after the adjustment of the model adaptive adjustment framework. By comprehensively evaluating the performance of the model adaptive adjustment framework on an independent test set, it is determined that the adjusted model adaptive adjustment framework can maintain or improve the accuracy and efficiency of anomaly detection in practical applications, and avoid overfitting, ensuring that the adjusted model adaptive adjustment framework can maintain efficient detection capabilities on both training data and test data. Through these detailed steps, it is ensured that the model adaptive adjustment framework can continuously optimize and adapt to the ever-changing network environment and threat types in practical applications, and improve the overall network security protection capabilities.

[0095] Step 5: Use the model adaptive adjustment framework optimized in step 4 to predict abnormal network traffic, and generate a detailed security report and corresponding response recommendations based on the prediction results. During this process, the security analyst will review and verify the report content based on the prediction results provided by the optimized model adaptive adjustment framework to ensure that all security recommendations are based on accurate and up-to-date data analysis. The analyst will also interpret and evaluate the key insights predicted by the optimized model adaptive adjustment framework to form the final security report and implementation recommendations.

[0096] In this step, a detailed security report is generated based on the prediction results, including detected abnormal events, possible attack types, impact scope, and recommended response measures. This report provides network administrators and security analysts with real-time security intelligence and data support, helping them to quickly take effective security protection measures. The specific steps are as follows:

[0097] Step 5.1: Use the model adaptive adjustment framework optimized in step 4 to predict abnormal network traffic. Summarize the prediction results and use the method in step 3 to perform correlation analysis to obtain analysis results, including identified abnormal events, attack patterns, and key affected assets, to form a comprehensive overview of the analysis results;

[0098] Step 5.2: Based on the summarized prediction results and correlation analysis results, use the report template or security information management system (SIEM) to automatically generate a detailed security report. Ensure that the report content includes not only the event summary, analysis details and impact assessment, but also targeted defense measures and response recommendations based on the prediction and actual analysis;

[0099] Step 5.3: Based on the analysis results and security impact assessment, propose specific response measures, which may include the application of emergency patches, configuration changes, isolation of affected systems, enhanced monitoring and other measures, as well as long-term security improvement recommendations;

[0100] Step 5.4 Distribute the security report to network administrators, security analysts, and senior management via email, security information and event management system (SIEM), or other automated tools. At the same time, activate the emergency response team or execute pre-planned response measures based on the recommendations in the report.

[0101] The embodiment of the present invention further provides a system for implementing the above-mentioned abnormal network traffic analysis method based on deep learning, comprising:

[0102] The traffic collection module is used to collect and preprocess network traffic data, and use a deep learning model to extract features from the preprocessed network traffic data;

[0103] A framework construction module is used to construct a model adaptive adjustment framework, and use the features extracted by the traffic collection module to train the model adaptive adjustment framework, and use the trained model adaptive adjustment framework to detect abnormal network traffic;

[0104] The abnormal event analysis module is used to analyze the correlation between abnormal events and potential attack patterns using abnormal correlation analysis technology in network traffic for the detected abnormal traffic;

[0105] A framework adjustment module is used to form a feedback data set according to the analysis results of the abnormal event analysis module, and optimize and adjust the model adaptability adjustment framework according to the feedback data set to obtain an optimized model adaptability adjustment framework;

[0106] The report generation module is used to predict network traffic using the optimized model adaptive adjustment framework and generate detailed security reports and corresponding response recommendations based on the prediction results.

[0107] The above are only preferred embodiments of the present invention, and are not intended to limit the implementation methods and protection scope of the present invention. Those skilled in the art should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the contents of the specification of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for analyzing abnormal network traffic based on deep learning, characterized in that: The steps include: Step 1: collect and preprocess network traffic data, and use a deep learning model to extract features from the preprocessed network traffic data; Step 2, constructing a model adaptive adjustment framework for detecting abnormal network traffic, using the features extracted in step 1 to train the model adaptive adjustment framework, and using the trained model adaptive adjustment framework to detect abnormal network traffic; Step 3: For the detected abnormal traffic, use the abnormal correlation analysis technology to analyze the correlation between abnormal events and potential attack patterns; Step 4: forming a feedback data set according to the analysis results of step 3, and optimizing and adjusting the model adaptability adjustment framework according to the feedback data set to obtain an optimized model adaptability adjustment framework; Step 5: Use the optimized model adaptive adjustment framework to predict network traffic anomalies, and generate a security report and corresponding response recommendations based on the prediction results; The specific implementation of step 1 includes: Step 1.1: Collect raw network traffic data from network devices and security systems, remove incomplete or damaged data records, and filter out data fields related to security analysis; Step 1.2: Convert network traffic data from different sources and formats into a unified format; Step 1.3, normalizing the network traffic data to obtain normalized network traffic data; Step 1.4 improves the ResNet-50 model and uses the improved ResNet-50 model to extract key features from the normalized network traffic data in step 1.3; Step 1.5 Use the improved ResNet-50 for feature selection; The improved ResNet-50 network structure includes: Convolutional layer, used to extract features; Pooling layer, used to reduce the spatial size of the feature map; Convolution blocks and identity blocks are stacked layers, where each convolution block includes three convolution layers and each identity block includes four convolution layers. The convolution block is used to further extract features, while the identity block is used to maintain the size and number of channels of the feature map unchanged.

2. According to the abnormal network traffic analysis method based on deep learning according to claim 1, it is characterized in that: The format unification method in step 1.2 includes: Unify timestamp format: convert all timestamps to ISO 8601 standard format; Unified IP address representation: Unify IP addresses into dotted decimal format; Data type conversion: convert the IP address field to a string type and the port number field to an integer type; Standardize the numerical data, including the flow size and the packet interval time. Protocol type encoding: Use one-hot encoding to convert the protocol type into an encoding vector.

3. The abnormal network traffic analysis method based on deep learning according to claim 1 is characterized in that: The model adaptation adjustment framework includes convolutional neural network (CNN), long short-term memory network (LSTM), and a combination of the two.

4. The abnormal network traffic analysis method based on deep learning according to claim 1 is characterized in that: The specific implementation of step 3 is as follows: Step 3.1 Based on the detected abnormal network traffic events, a graph is constructed according to the event features and predefined association rules, where nodes represent individual abnormal events and edges between nodes represent potential associations between events; Step 3.2: Use graph algorithms to analyze the graph constructed in step 3.

1. Use the shortest path algorithm to find the shortest path between abnormal events, identify key attack paths and diffusion paths, and use the community discovery algorithm to identify subgroups and communities in the graph, revealing the association patterns between different attack events. Step 3.3: Based on the results of the graph algorithm analysis in step 3.2, a specific security impact assessment is performed on each identified attack chain or association pattern; Step 3.4 provides specific defense measures and response suggestions for each identified attack mode or link based on the security impact assessment results in step 3.

3.

5. The abnormal network traffic analysis method based on deep learning according to claim 4 is characterized in that: The specific implementation of step 3.3 is: Analyze the key nodes and paths in each attack chain, determine the affected assets, and assess the risks that may be caused by the attack on these assets. At the same time, consider the diffusion path of each attack chain and quantitatively assess its potential for propagation throughout the network. The quantitative assessment is carried out by setting risk scores, assigning a risk level from low to high to each key node and path, and estimating the direct and indirect losses that may be caused by the attack, including the cost of data loss and equipment damage.

6. The abnormal network traffic analysis method based on deep learning according to claim 1 is characterized in that: The specific implementation of step 4 is as follows: Step 4.1 Summarize the results of anomaly detection and association analysis, including the detection accuracy, false positive rate, false negative rate, and the accuracy and practicality evaluation of association analysis, to form a comprehensive feedback data set; Step 4.2 After aggregating the feedback data, evaluate the performance of the model adaptive adjustment framework based on the feedback data; Step 4.3: According to the performance evaluation results in step 4.2, the parameters and structure of the model adaptive adjustment framework are adjusted, and the detected abnormal traffic samples and the correlation analysis results are used as new training data to continue training the model adaptive adjustment framework to obtain a fine-tuned model adaptive adjustment framework; Step 4.4 evaluates the detection accuracy and generalization ability of the fine-tuned model adaptation adjustment framework to obtain the optimized model adaptation adjustment framework.

7. The abnormal network traffic analysis method based on deep learning according to claim 1 is characterized in that: Step 5 specifically includes: Step 5.1 Summarize all key information obtained in the previous analysis steps, organize and compile the data and analysis results generated in step 4, including the identified abnormal events, attack patterns, and key affected assets, to form a comprehensive overview of the analysis results; Step 5.2 Based on the summarized analysis results, use a report template or security information management system to automatically generate a report, ensuring that the report content includes an incident summary, analysis details, impact assessment, and targeted defense measures and response recommendations; Step 5.3 Based on the analysis results and security impact assessment, propose specific response measures, including emergency patch application, configuration changes, isolation of affected systems, enhanced monitoring, and long-term security improvement recommendations; Step 5.4 Distribute the security report to network administrators, security analysts, and senior management via email, security information and event management systems, or other automated tools, and activate the emergency response team or execute pre-planned response measures based on the recommendations in the report.

8. A system for implementing the abnormal network traffic analysis method based on deep learning according to any one of claims 1 to 7, characterized in that: include: The traffic collection module is used to collect and preprocess network traffic data, and use a deep learning model to extract features from the preprocessed network traffic data; A framework construction module is used to construct a model adaptive adjustment framework, and use the features extracted by the traffic collection module to train the model adaptive adjustment framework, and use the trained model adaptive adjustment framework to detect abnormal network traffic; The abnormal event analysis module is used to analyze the correlation between abnormal events and potential attack patterns using abnormal correlation analysis technology in network traffic for the detected abnormal traffic; A framework adjustment module is used to form a feedback data set according to the analysis results of the abnormal event analysis module, and optimize and adjust the model adaptability adjustment framework according to the feedback data set to obtain an optimized model adaptability adjustment framework; The report generation module is used to predict network traffic using the optimized model adaptive adjustment framework and generate security reports and corresponding response recommendations based on the prediction results.

Images