A deep learning-based abnormal network traffic detection method, system, device and medium

Abstract

The application provides a kind of based on deep learning's abnormal network flow detection method, system, equipment and medium, it is related to network security protection technical field, including setting discrete data processing rule;Capture network flow data and carry out data splitting and length modification, then convert into network flow grayscale image;Use convolutional neural network to construct and train flow detection model;The network flow grayscale image is input into flow detection model, and the output network flow anomaly probability.The application can improve the network flow data monitoring efficiency, improve the identification accuracy of abnormal flow, reduce the false alarm and leakage report of abnormal flow, reduce the dependence on human resources, improve the overall performance and adaptability of abnormal network flow detection system.

Description

Technical Field

[0001] This invention relates to the field of network security protection technology, specifically to a method, system, device, and medium for detecting abnormal network traffic based on deep learning. Background Technology

[0002] With the rapid development of network technology, network traffic is growing exponentially, and network attack methods are becoming increasingly complex, with new threats such as DDoS attacks and malware propagation constantly emerging. Traditional security protection methods are struggling to cope with the dynamically changing network environment, necessitating an efficient and adaptive abnormal traffic detection technology to enhance network security protection capabilities.

[0003] Existing technologies mostly employ rule-based matching or shallow machine learning models (such as support vector machines and random forests) for abnormal traffic detection. The specific process involves manually defining features (such as packet size, protocol type, and port number) to extract statistical characteristics of network traffic, which are then input into a classification model for anomaly determination. Some studies attempt to convert traffic data into time-series signals and use recurrent neural networks (RNNs) or long short-term memory networks (LSTMs) to extract time-series features.

[0004] However, existing methods for anomaly traffic detection based on rule matching or shallow machine learning models often rely on manual feature extraction, making it difficult to capture deep-seated correlations in traffic data, resulting in low detection accuracy and high false alarm rates. Specifically, traditional machine learning models lack generalization ability and cannot adapt to complex and ever-changing network environments and unknown attack patterns; methods based on time-series models are inefficient at processing unstructured traffic data, making it difficult to meet real-time detection requirements. Summary of the Invention

[0005] To address the technical problems of existing methods for anomaly traffic detection based on rule matching or shallow machine learning models, which rely on manual feature extraction, making it difficult to capture deep-level correlation features in traffic data, resulting in low detection accuracy, high false alarm rate, insufficient generalization ability, and low data processing efficiency, this invention provides a deep learning-based method, system, device, and medium for anomaly network traffic detection. This method can improve the efficiency of network traffic data monitoring, enhance the accuracy of anomaly traffic identification, reduce false alarms and missed alarms, reduce reliance on human resources, and improve the overall performance and adaptability of the anomaly network traffic detection system.

[0006] In a first aspect, the present invention provides a method for detecting abnormal network traffic based on deep learning, comprising the following steps: S1. Set discrete data processing rules. The discrete data processing rules are to perform one-hot encoding on discrete data to generate continuous feature vectors. S2. Capture network traffic data and split the data into short stream data files. Trim the length of the short stream data files to make the length of the short stream data files uniform. Then, convert all short stream data files belonging to the same network traffic data into a single network traffic grayscale image. Among them, after capturing network traffic data, discrete data processing rules are used to process the discrete data in the network traffic data, and then the data is split. S3. Construct and train a traffic detection model using a convolutional neural network. The traffic detection model includes an input module, a feature extraction module, and a classification decision module. The feature extraction module includes at least two cascaded feature extraction units, each of which contains an attention module, a residual convolution module, and a skip connection; The classification decision module includes a pooling layer and a classification layer. The classification layer includes a fully connected layer and an activation function. The output of the traffic detection model is the probability of network traffic anomalies. , [0,1]; S4. Input the grayscale image of network traffic into the traffic detection model and output the probability of network traffic anomalies.

[0007] It should be further noted that in step S1, the discrete data includes protocol type, port number, and traffic attributes encoded with integers.

[0008] It should be further explained that the specific operation of step S2 is as follows: S201. Use packet capture functions to capture network traffic data, use discrete data processing rules to process discrete data in the network traffic data, and then save the network traffic data as a pcap file; S202. Use a splitting tool to split the pcap file of network traffic data according to the five-tuple method, and split the pcap file of network traffic data into short stream data files. The five-tuple includes source IP, source port, destination IP, destination port, and protocol type. S203. Trim the length of the short stream data file to make the number of bytes in all short streams consistent, discarding extra bytes and padding insufficient bytes with zeros; S204. Convert each byte in the short-stream data file into a pixel. The grayscale value of each pixel is 0-255. The higher the byte value, the higher the corresponding grayscale value, and the lower the byte value, the lower the corresponding grayscale value. Finally, convert all short-stream data files belonging to the same network traffic data into a network traffic grayscale image.

[0009] It should be further noted that the splitting tool includes either the splitcap tool or the editcap tool.

[0010] It should be further noted that the attention mechanism of the attention module in the feature extraction unit of step S3 is described by the following formula:

[0011] in:

[0012] In the formula, , , The linear mapping parameter for the input vector is used to perform feature mapping on the original input vector. This represents the alignment function, used to measure... and Similarity between them; Used to convert similarity into a weighted form of 0-1; This is the output of the attention module; The convolutional residual module consists of three convolutional layers. The output of the attention module is used as the input to the first convolutional layer, and the input to the remaining convolutional layers is the output of the previous convolutional layer. The overall expression for the three convolutional layers is:

[0013] In the formula, C is the output of the third convolutional layer; The skip connection adds the input of the feature extraction unit directly to the output of the third convolutional layer, thus achieving an identity mapping. The expression is: .

[0014] It should be further noted that the activation function is the Sigmoid function.

[0015] It should be further explained that step S5 is also included: determining whether network traffic data belongs to abnormal network traffic by using a manually set abnormal probability threshold. The determination criterion is: if the abnormal probability of the network traffic corresponding to the network traffic data is greater than or equal to the abnormal probability threshold, then the network traffic data belongs to abnormal network traffic.

[0016] It should be further noted that the anomaly probability threshold is 0.5.

[0017] Secondly, the present invention provides a deep learning-based abnormal network traffic detection system for implementing the aforementioned deep learning-based abnormal network traffic detection method, comprising: The data processing rule setting module is used to set rules for discrete data processing; The network traffic data capture module is used to capture network traffic data. The data processing module is used to process discrete data in network traffic data using discrete data processing rules, and to split network traffic data into short stream data files, trim the length of the short stream data files to make the length of the short stream data files uniform, and then convert all short stream data files belonging to the same network traffic data into a network traffic grayscale image. The model building and training module is used to build and train traffic detection models using convolutional neural networks; The detection module is used to input grayscale images of network traffic into the traffic detection model and output the probability of network traffic anomalies.

[0018] Thirdly, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described deep learning-based abnormal network traffic detection method.

[0019] Fourthly, the present invention provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described deep learning-based abnormal network traffic detection method.

[0020] The beneficial effects of this invention are as follows: 1. The deep learning-based abnormal network traffic detection method provided by this invention captures network traffic data after setting discrete data processing rules, performs data splitting and length trimming, converts the network traffic data into a grayscale image, and then uses a convolutional neural network to build and train a traffic detection model. The grayscale image is input into the traffic detection model, which outputs the probability of network traffic anomalies. By leveraging the powerful feature extraction capabilities of convolutional neural networks, deep and complex feature patterns are mined from the image-based network traffic data, thereby improving the efficiency of network traffic data monitoring, increasing the accuracy of abnormal traffic identification, and reducing false alarms and missed alarms.

[0021] 2. This invention utilizes convolutional neural networks to construct and train a traffic detection model for detecting network traffic data. The trained traffic detection model is robust, adaptable to different network environments and different types of network traffic data, and can effectively detect unknown abnormal behavior patterns, thereby improving the overall performance and adaptability of the abnormal network traffic detection system.

[0022] 3. Compared with existing methods for abnormal traffic detection based on rule matching or shallow machine learning models, the present invention can significantly reduce the dependence on human resources, reduce the burden on operation and maintenance personnel, and thus reduce the overall operation and maintenance costs.

[0023] 4. This invention can promptly detect and respond to abnormal behaviors in the network, such as DDoS attacks and malware propagation, effectively curbing the spread of network threats, protecting user data and system security, and improving the overall security level of the network.

[0024] 5. The configuration and parameters of the traffic detection model of the present invention can be adjusted according to the actual needs of users and the characteristics of the network environment, so that the abnormal network traffic detection system can more flexibly adapt to different application scenarios and needs. Attached Figure Description

[0025] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart of an abnormal network traffic detection method based on deep learning in one embodiment of the present invention.

[0027] Figure 2 This is a grayscale image of network traffic generated according to different types of network traffic in one embodiment of the present invention.

[0028] Figure 3 This is a schematic block diagram of an abnormal network traffic detection system based on deep learning in one embodiment of the present invention.

[0029] Figure 4 This is a schematic diagram of the hardware structure of an electronic device in one embodiment of the present invention. Detailed Implementation

[0030] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings of the specific embodiments. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0031] This application relates to a deep learning-based method for detecting abnormal network traffic, primarily targeting the field of network security protection technology. The technical solution includes setting discrete data processing rules; capturing network traffic data, performing data splitting and length trimming, and then converting it into a grayscale image of network traffic; constructing and training a traffic detection model using a convolutional neural network; inputting the grayscale image of network traffic into the traffic detection model, and outputting the probability of abnormal network traffic. This invention captures network traffic data by setting discrete data processing rules, performing data splitting and length trimming, converting the network traffic data into a grayscale image of network traffic, then constructing and training a traffic detection model using a convolutional neural network, inputting the grayscale image of network traffic into the traffic detection model, and outputting the probability of abnormal network traffic. By leveraging the powerful feature extraction capabilities of convolutional neural networks (CNNs), deep and complex feature patterns are mined from image-based network traffic data, thereby improving the efficiency of network traffic data monitoring and the accuracy of abnormal traffic identification, reducing false alarms and missed alarms. A traffic detection model is built and trained using CNNs to detect network traffic data. The trained traffic detection model is robust, adaptable to different network environments and different types of network traffic data, and can effectively detect unknown abnormal behavior patterns, improving the overall performance and adaptability of the abnormal network traffic detection system. Compared to existing methods based on rule matching or shallow machine learning models for abnormal traffic detection, this invention significantly reduces reliance on human resources, reduces the burden on maintenance personnel, and thus lowers overall maintenance costs. It can promptly detect and respond to abnormal behaviors in the network, such as DDoS attacks and malware propagation, effectively curbing the spread of network threats, protecting user data and system security, and improving the overall network security level. The configuration and parameters of the traffic detection model can be adjusted according to the actual needs of users and the characteristics of the network environment, making the abnormal network traffic detection system more flexible and adaptable to different application scenarios and requirements.

[0032] The deep learning-based abnormal network traffic detection method disclosed in this application addresses the technical problems of existing abnormal traffic detection methods based on rule matching or shallow machine learning models, which rely on manual feature extraction, making it difficult to capture deep-level correlation features in traffic data, resulting in low detection accuracy, high false alarm rate, insufficient generalization ability, and low data processing efficiency.

[0033] The following describes in detail the deep learning-based abnormal network traffic detection method of this application. Specific details, such as particular system architectures and techniques, are presented for illustrative purposes and not for limitation, to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application can also be implemented in other embodiments without these specific details.

[0034] In the deep learning-based abnormal network traffic detection method disclosed in this application, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or sets thereof. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0035] To facilitate a clear description of the technical solutions of this application, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" do not necessarily imply that they are different.

[0036] The terms "one embodiment" or "some embodiments" used in this application mean that one or more embodiments of this application include the specific features, structures, or characteristics described in that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this application do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] The deep learning-based abnormal network traffic detection method provided in this embodiment of the invention is executed by a computer device, and correspondingly, the deep learning-based abnormal network traffic detection system runs on the computer device.

[0039] Figure 1 This is a flowchart of an abnormal network traffic detection method based on deep learning, according to an embodiment of the present invention. Figure 1 The implementing entity can be a deep learning-based abnormal network traffic detection system. Depending on the specific requirements, the order of the steps in this flowchart can be changed, and some steps can be omitted.

[0040] like Figure 1 As shown, this deep learning-based method for detecting abnormal network traffic includes: Step S1: Set discrete data processing rules. The discrete data processing rules are to perform one-hot encoding on discrete data to generate continuous feature vectors.

[0041] By setting discrete data processing rules and performing one-hot encoding on discrete data to generate continuous feature vectors, the data becomes more suitable for deep learning model processing. This preserves the original data features, improves data usability, lays the foundation for accurate subsequent analysis of network traffic data, and enhances the accuracy of feature extraction.

[0042] In some specific embodiments, discrete data includes protocol type, port number, and integer-encoded traffic attributes.

[0043] Determining the range of discrete data provides a clear basis for subsequent data processing, ensuring the integrity and accuracy of data processing, and helping to extract features more accurately from network traffic data, thereby improving the accuracy of abnormal traffic detection.

[0044] Step S2: Capture network traffic data and split the data into short stream data files. Trim the length of the short stream data files to make the length of the short stream data files uniform. Then, convert all short stream data files belonging to the same network traffic data into a single network traffic grayscale image. This process involves capturing network traffic data, processing the discrete data within it using discrete data processing rules, and then splitting the data.

[0045] Transforming network traffic data into an image format suitable for convolutional neural network processing can fully leverage the advantages of convolutional neural networks in image feature extraction, uncovering deep and complex feature patterns, thereby improving the efficiency of network traffic data monitoring and enhancing the accuracy of abnormal traffic identification.

[0046] In some specific embodiments, step S2 is performed as follows: S201. Use packet capture functions to capture network traffic data, use discrete data processing rules to process discrete data in the network traffic data, and then save the network traffic data as a pcap file; S202. Use a splitting tool to split the pcap file of network traffic data according to the five-tuple method, and split the pcap file of network traffic data into short stream data files. The five-tuple includes source IP, source port, destination IP, destination port, and protocol type. S203. Trim the length of the short stream data file to make the number of bytes in all short streams consistent, discarding extra bytes and padding insufficient bytes with zeros; S204. Convert each byte in the short-stream data file into a pixel. The grayscale value of each pixel is 0-255. The higher the byte value, the higher the corresponding grayscale value, and the lower the byte value, the lower the corresponding grayscale value. Finally, convert all short-stream data files belonging to the same network traffic data into a network traffic grayscale image.

[0047] By standardizing the process of capturing, splitting, trimming, and converting network traffic data into grayscale images, the standardization and consistency of data processing can be guaranteed, enabling the data to be better adapted to subsequent convolutional neural network processing, improving the quality of input data for the detection model, and thus enhancing the reliability of abnormal traffic detection.

[0048] In some specific embodiments, the splitting tool includes either the splitcap tool or the editcap tool.

[0049] Specifying the scope of splitting tools provides concrete implementation methods for data splitting operations, making it easier for technicians to select appropriate tools for data processing, ensuring the efficient execution of data splitting work, and helping to improve the efficiency of the entire abnormal traffic detection process.

[0050] Step S3: Construct and train a traffic detection model using a convolutional neural network. The traffic detection model includes an input module, a feature extraction module, and a classification decision module. The feature extraction module includes at least two cascaded feature extraction units, each of which contains an attention module, a residual convolution module, and a skip connection; The classification decision module includes a pooling layer and a classification layer. The classification layer includes a fully connected layer and an activation function. The output of the traffic detection model is the probability of network traffic anomalies. , [0,1] .

[0051] In some specific embodiments, the attention mechanism of the attention module in the feature extraction unit is described by the following formula:

[0052] in:

[0053] In the formula, , , The linear mapping parameter for the input vector is used to perform feature mapping on the original input vector. This represents the alignment function, used to measure... and Similarity between them; Used to convert similarity into a weighted form of 0-1; This is the output of the attention module; The convolutional residual module consists of three convolutional layers. The output of the attention module is used as the input to the first convolutional layer, and the input to the remaining convolutional layers is the output of the previous convolutional layer. The overall expression for the three convolutional layers is:

[0054] In the formula, C is the output of the third convolutional layer; The skip connection adds the input of the feature extraction unit directly to the output of the third convolutional layer, thus achieving an identity mapping. The expression is: .

[0055] By providing formulas for the attention module, convolutional residual module, and skip connections, this paper explains in detail the working principles of key components within the model. This helps to deepen the understanding of the model's feature extraction and processing mechanisms, providing theoretical support for optimizing the model structure and improving its performance. This enables the detection model to more effectively mine deep features in network traffic data.

[0056] In some specific embodiments, the activation function is the Sigmoid function.

[0057] In a model, Sigmoid can map the output to the [0,1] interval, which can conveniently and intuitively represent the probability of network traffic anomalies. Furthermore, its characteristics help with model training and convergence, improving model stability and detection accuracy.

[0058] In some specific embodiments, step S5 is also included: determining whether network traffic data belongs to abnormal network traffic by using a manually set abnormal probability threshold. The determination criterion is: if the abnormal probability of the network traffic corresponding to the network traffic data is greater than or equal to the abnormal probability threshold, then the network traffic data belongs to abnormal network traffic.

[0059] By setting an anomaly probability threshold to determine abnormal network traffic, a clear standard is provided for judging abnormal traffic, making the detection results operable, facilitating the timely detection and handling of abnormal traffic, and enhancing network security protection capabilities.

[0060] In some specific embodiments, the anomaly probability threshold is 0.5.

[0061] Among them, the attention module in the feature extraction module can automatically focus on key features and enhance the weight of important features; the residual convolution module extracts features at different levels step by step through multiple convolutional layers, which can deepen the mining of data features; skip connections can avoid the gradient vanishing problem, ensure the stability and accuracy of model training, and enable the model to learn the features of network traffic data more comprehensively and accurately; pooling layers can reduce data dimensionality and reduce computational load; fully connected layers comprehensively process features; and activation functions output the probability of network traffic anomalies, making it convenient to intuitively judge whether the traffic is abnormal, which can improve the interpretability and practicality of detection results; the trained model has strong robustness, can adapt to different network environments and types of network traffic data, effectively detect unknown abnormal behavior patterns, and improve the overall performance and adaptability of the abnormal network traffic detection system.

[0062] Step S4: Input the grayscale image of network traffic into the traffic detection model and output the probability of network traffic anomalies.

[0063] Based on the feature learning and classification capabilities of the traffic detection model, it is possible to quickly and accurately determine abnormal network traffic, promptly detect and respond to abnormal behaviors in the network, protect user data and system security, and improve the overall security level of the network.

[0064] In one specific embodiment, the deep learning-based abnormal network traffic detection method includes: Step S1: Set discrete data processing rules. The discrete data processing rules are to perform one-hot encoding on discrete data to generate continuous feature vectors. Discrete data includes protocol type, port number, and traffic attributes encoded by integers. Step S2: Capture network traffic data and split the data into short stream data files. Trim the length of each short stream data file to ensure uniform length. Then, convert all short stream data files belonging to the same network traffic type into a single network traffic grayscale image. The network traffic grayscale images formed according to different types of network traffic are as follows: Figure 2 As shown; Among them, after capturing network traffic data, discrete data processing rules are used to process the discrete data in the network traffic data, and then the data is split. The specific steps are as follows: S201. Use packet capture functions to capture network traffic data, use discrete data processing rules to process discrete data in the network traffic data, and then save the network traffic data as a pcap file; S202. Use a splitting tool to split the pcap file of network traffic data according to the five-tuple method, splitting the pcap file of network traffic data into short stream data files. The five-tuple includes source IP, source port, destination IP, destination port, and protocol type. The splitting tool includes either the splitcap tool or the editcap tool.

[0065] S203. Trim the length of the short stream data file to make the number of bytes in all short streams consistent, discarding extra bytes and padding insufficient bytes with zeros; S204. Convert each byte in the short-stream data file into a pixel. The grayscale value of each pixel is 0-255. The higher the byte value, the higher the corresponding grayscale value, and the lower the byte value, the lower the corresponding grayscale value. Finally, convert all short-stream data files belonging to the same network traffic data into a network traffic grayscale image. Step S3: Construct and train a traffic detection model using a convolutional neural network. The traffic detection model includes an input module, a feature extraction module, and a classification decision module. The feature extraction module includes four cascaded feature extraction units, each of which contains an attention module, a residual convolution module, and a skip connection; In the feature extraction unit, the attention mechanism of the attention module is described by the following formula:

[0066] in:

[0067] In the formula, , , The linear mapping parameter for the input vector is used to perform feature mapping on the original input vector. This represents the alignment function, used to measure... and Similarity between them; Used to convert similarity into a weighted form of 0-1; This is the output of the attention module; The convolutional residual module consists of three convolutional layers, each configured as follows: Layer 1: 3×3 convolution, 64 filters, stride 1, activation function is ReLU; Layer 2: 3×3 convolution, 64 filters, stride 1, activation function is ReLU; Layer 3: 1×1 convolution, 64 filters, stride 1, used to adjust the number of channels; The output of the attention module is used as the input to the first convolutional layer, and the input to the remaining convolutional layers is the output of the previous convolutional layer. The overall expression for the three convolutional layers is:

[0068] In the formula, C is the output of the third convolutional layer; The skip connection adds the input of the feature extraction unit directly to the output of the third convolutional layer, thus achieving an identity mapping. The expression is: ; The classification decision module includes a pooling layer and a classification layer. The pooling layer performs average pooling on the output of the feature extraction unit. The classification layer includes a fully connected layer and an activation function, which is the Sigmoid function. The output of the traffic detection model is the probability of network traffic anomalies. , [0,1] ; The training steps for the traffic detection model are as follows: S301, A large number of network traffic grayscale images are obtained using the method in step S2, forming a network traffic grayscale image set. The network traffic grayscale image set includes network traffic grayscale images corresponding to normal traffic and abnormal traffic. Abnormal traffic includes network traffic generated when DDoS and / or port scanning occur. Random data augmentation operations are performed on the grayscale images of network traffic in the set of grayscale images of network traffic. The data augmentation operations include horizontal flipping, ±10% translation and Gaussian noise injection. The randomly augmented set of grayscale images of network traffic is divided into training, validation, and test sets in a ratio of 7:2:1. S302, set the optimizer to AdamW, and the initial learning rate to 1×10⁻⁶. -4 The weight decays to 1×10 −5 ; The loss function is set to binary cross-entropy loss, and the expression for the loss function is:

[0069] Set the batch size to 128, the training period to 100, and the early stop mechanism to terminate training if the validation set loss does not decrease for 5 consecutive times. The learning rate scheduling uses a cosine annealing strategy with a period of 10 epochs and a minimum learning rate of 1×10. -6 ; S303, Initialize the model, input the grayscale image of network traffic in the training set, calculate layer by layer through the four feature extraction units in the feature extraction module, and output the anomaly probability; S304 uses a loss function to calculate the loss and update the parameters of the feature extraction unit, setting the gradient clipping threshold to 5.0 during the process; S305, after each epoch, evaluate the F1 score on the validation set, and after all epochs, save the traffic detection model with the best evaluation result as the best model. S306. Load the best model, calculate the accuracy, recall, and AUC-ROC on the test set. If the manually set threshold is not met, repeat steps S303-S306 until the manually set threshold is met and then stop to obtain the trained traffic detection model. Step S4: Input the grayscale image of network traffic into the traffic detection model and output the probability of network traffic anomalies. Step S5: Determine whether network traffic data belongs to abnormal network traffic by using a manually set abnormal probability threshold. The abnormal probability threshold is 0.5. The determination criterion is: if the abnormal probability of the network traffic corresponding to the network traffic data is greater than or equal to the abnormal probability threshold, then the network traffic data belongs to abnormal network traffic.

[0070] The following are embodiments of the abnormal network traffic detection system based on deep learning provided in this disclosure. This active power load reduction optimization system and the abnormal network traffic detection method based on deep learning in the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the abnormal network traffic detection system based on deep learning, please refer to the embodiments of the abnormal network traffic detection method based on deep learning described above.

[0071] Mobile terminals implementing various embodiments of the present invention will now be described with reference to the accompanying drawings. In the following description, suffixes such as "module," "part," or "unit" used to denote elements are used only for the convenience of illustrating the embodiments of the present invention and have no specific meaning in themselves. Therefore, "module" and "part" can be used interchangeably.

[0072] like Figure 3 As shown, the deep learning-based abnormal network traffic detection system includes: The data processing rule setting module is used to set rules for discrete data processing; The network traffic data capture module is used to capture network traffic data. The data processing module is used to process discrete data in network traffic data using discrete data processing rules, and to split network traffic data into short stream data files, trim the length of the short stream data files to make the length of the short stream data files uniform, and then convert all short stream data files belonging to the same network traffic data into a network traffic grayscale image. The model building and training module is used to build and train traffic detection models using convolutional neural networks; The detection module is used to input grayscale images of network traffic into the traffic detection model and output the probability of network traffic anomalies.

[0073] The abnormal network traffic detection system in this embodiment is used to implement an abnormal network traffic detection method based on deep learning. The steps include: S1. Set discrete data processing rules. The discrete data processing rules are to perform one-hot encoding on discrete data to generate continuous feature vectors. S2. Capture network traffic data and split the data into short stream data files. Trim the length of the short stream data files to make the length of the short stream data files uniform. Then, convert all short stream data files belonging to the same network traffic data into a single network traffic grayscale image. Among them, after capturing network traffic data, discrete data processing rules are used to process the discrete data in the network traffic data, and then the data is split. S3. Construct and train a traffic detection model using a convolutional neural network. The traffic detection model includes an input module, a feature extraction module, and a classification decision module. The feature extraction module includes at least two cascaded feature extraction units, each of which contains an attention module, a residual convolution module, and a skip connection; The classification decision module includes a pooling layer and a classification layer. The classification layer includes a fully connected layer and an activation function. The output of the traffic detection model is the probability of network traffic anomalies. , [0,1] ; S4. Input the grayscale image of network traffic into the traffic detection model and output the probability of network traffic anomalies.

[0074] This application also provides an electronic device for implementing various embodiments of the present invention, the electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor.

[0075] Those skilled in the art will understand that the electronic device structure involved in the embodiments of the present invention does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0076] Figure 4 A schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention.

[0077] Electronic devices include, but are not limited to, components such as processors and memory. Those skilled in the art will understand that the electronic device structures described in the embodiments of this invention do not constitute a limitation on the electronic device. An electronic device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0078] In embodiments of the present invention, electronic devices include, but are not limited to, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments described and / or claimed herein.

[0079] In this application embodiment, the processor can be implemented using at least one of an Application-Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a processor, a controller, a microcontroller, a microprocessor, or an electronic unit designed to perform the functions described herein. In some cases, such implementations can be implemented within a controller. For software implementations, implementations such as processes or functions can be implemented with separate software modules that allow the performance of at least one function or operation. The software code can be implemented by a software application (or program) written in any suitable programming language, and the software code can be stored in memory and executed by the controller.

[0080] In addition, the electronic device includes some functional modules not shown, which will not be described in detail here.

[0081] Those skilled in the art will understand that the various aspects of the electronic device provided in this application can be implemented as a system, method, or program product. Therefore, the various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0082] This application also provides a storage medium storing a program product capable of implementing a deep learning-based abnormal network traffic detection method. In some possible implementations, various aspects of this disclosure can also be implemented as a program product comprising program code that, when run on a terminal device, causes the terminal device to perform the steps described in the foregoing "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0083] The storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example,, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0084] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting abnormal network traffic based on deep learning, characterized by the following steps: include: S1. Set discrete data processing rules. The discrete data processing rules are to perform one-hot encoding on discrete data to generate continuous feature vectors. S2. Capture network traffic data and split the data into short stream data files. Trim the length of the short stream data files to make the length of the short stream data files uniform. Then, convert all short stream data files belonging to the same network traffic data into a single network traffic grayscale image. Among them, after capturing network traffic data, discrete data processing rules are used to process the discrete data in the network traffic data, and then the data is split. S3. Construct and train a traffic detection model using a convolutional neural network. The traffic detection model includes an input module, a feature extraction module, and a classification decision module. The feature extraction module includes at least two cascaded feature extraction units, each of which contains an attention module, a residual convolution module, and a skip connection; The classification decision module includes a pooling layer and a classification layer. The classification layer includes a fully connected layer and an activation function. The output of the traffic detection model is the probability of network traffic anomalies. , [0,1]; S4. Input the grayscale image of network traffic into the traffic detection model and output the probability of network traffic anomalies; In the feature extraction unit of step S3, the attention mechanism of the attention module is described by the following formula: in: In the formula, , , The linear mapping parameter for the input vector is used to perform feature mapping on the original input vector. This represents the alignment function, used to measure... and Similarity between them; Used to convert similarity into a weighted form of 0-1; This is the output of the attention module; The convolutional residual module consists of three convolutional layers. The output of the attention module is used as the input to the first convolutional layer, and the input to the remaining convolutional layers is the output of the previous convolutional layer. The overall expression for the three convolutional layers is: In the formula, C is the output of the third convolutional layer; The skip connection adds the input of the feature extraction unit directly to the output of the third convolutional layer, thus achieving an identity mapping. The expression is: ； The three convolutional layers of the convolutional residual module are configured as follows: Layer 1: 3×3 convolution, 64 filters, stride 1, activation function is ReLU; Layer 2: 3×3 convolution, 64 filters, stride 1, activation function is ReLU; Layer 3: 1×1 convolution, 64 filters, stride 1, used to adjust the number of channels; The training steps for the traffic detection model are as follows: S301, A large number of network traffic grayscale images are obtained using the method in step S2, forming a network traffic grayscale image set. The network traffic grayscale image set includes network traffic grayscale images corresponding to normal traffic and abnormal traffic. Abnormal traffic includes network traffic generated when DDoS and / or port scanning occur. Random data augmentation operations are performed on the grayscale images of network traffic in the set of grayscale images of network traffic. The data augmentation operations include horizontal flipping, ±10% translation and Gaussian noise injection. The randomly augmented set of grayscale images of network traffic is divided into training, validation, and test sets in a ratio of 7:2:

1. S302, set the optimizer to AdamW, and the initial learning rate to 1×10⁻⁶. -4 The weight decays to 1×10 -5 ; The loss function is set to binary cross-entropy loss, and the expression for the loss function is: Set the batch size to 128, the training period to 100, and the early stop mechanism to terminate training if the validation set loss does not decrease for 5 consecutive times. The learning rate scheduling uses a cosine annealing strategy with a period of 10 epochs and a minimum learning rate of 1×10. -6 ; S303, Initialize the model, input the grayscale image of network traffic in the training set, calculate layer by layer through the four feature extraction units in the feature extraction module, and output the anomaly probability; S304 uses a loss function to calculate the loss and update the parameters of the feature extraction unit, setting the gradient clipping threshold to 5.0 during the process; S305. After each epoch, the F1 score is evaluated on the validation set. After all epochs are completed, the traffic detection model with the best evaluation result is saved as the best model. S306. Load the best model, calculate the accuracy, recall, and AUC-ROC on the test set. If the manually set threshold is not met, repeat steps S303-S306 until the manually set threshold is met, and then stop to obtain the trained traffic detection model.

2. The abnormal network traffic detection method as described in claim 1, characterized in that, In step S1, the discrete data includes protocol type, port number, and integer-encoded traffic attributes.

3. The abnormal network traffic detection method as described in claim 1, characterized in that, The specific operation of step S2 is as follows: S201. Use packet capture functions to capture network traffic data, use discrete data processing rules to process discrete data in the network traffic data, and then save the network traffic data as a pcap file; S202. Use a splitting tool to split the pcap file of network traffic data according to the five-tuple method, and split the pcap file of network traffic data into short stream data files. The five-tuple includes source IP, source port, destination IP, destination port, and protocol type. S203. Trim the length of the short stream data file to make the number of bytes in all short streams consistent, discarding extra bytes and padding insufficient bytes with zeros; S204. Convert each byte in the short-stream data file into a pixel. The grayscale value of each pixel is 0-255. The higher the byte value, the higher the corresponding grayscale value, and the lower the byte value, the lower the corresponding grayscale value. Finally, convert all short-stream data files belonging to the same network traffic data into a network traffic grayscale image.

4. The abnormal network traffic detection method as described in claim 3, characterized in that, Splitting tools include either splitcap or editcap.

5. The abnormal network traffic detection method as described in claim 1, characterized in that, The activation function is the Sigmoid function.

6. The abnormal network traffic detection method as described in claim 1, characterized in that, It also includes step S5: determining whether network traffic data belongs to abnormal network traffic by using a manually set abnormal probability threshold. The determination criterion is: if the abnormal probability of the network traffic corresponding to the network traffic data is greater than or equal to the abnormal probability threshold, then the network traffic data belongs to abnormal network traffic.

7. A deep learning-based abnormal network traffic detection system, characterized in that, To implement the abnormal network traffic detection method as described in any one of claims 1-6, the method includes: The data processing rule setting module is used to set rules for discrete data processing; The network traffic data capture module is used to capture network traffic data. The data processing module is used to process discrete data in network traffic data using discrete data processing rules, and to split network traffic data into short stream data files, trim the length of the short stream data files to make the length of the short stream data files uniform, and then convert all short stream data files belonging to the same network traffic data into a network traffic grayscale image. The model building and training module is used to build and train traffic detection models using convolutional neural networks; The detection module is used to input the grayscale image of network traffic into the traffic detection model and output the probability of network traffic anomalies. In the feature extraction unit of step S3, the attention mechanism of the attention module is described by the following formula: in: In the formula, , , The linear mapping parameter for the input vector is used to perform feature mapping on the original input vector. This represents the alignment function, used to measure... and Similarity between them; Used to convert similarity into a weighted form of 0-1; This is the output of the attention module; The convolutional residual module consists of three convolutional layers. The output of the attention module is used as the input to the first convolutional layer, and the input to the remaining convolutional layers is the output of the previous convolutional layer. The overall expression for the three convolutional layers is: In the formula, C is the output of the third convolutional layer; The skip connection adds the input of the feature extraction unit directly to the output of the third convolutional layer, thus achieving an identity mapping. The expression is: ； The three convolutional layers of the convolutional residual module are configured as follows: Layer 1: 3×3 convolution, 64 filters, stride 1, activation function is ReLU; Layer 2: 3×3 convolution, 64 filters, stride 1, activation function is ReLU; Layer 3: 1×1 convolution, 64 filters, stride 1, used to adjust the number of channels; The training steps for the traffic detection model are as follows: S301, A large number of network traffic grayscale images are obtained using the method in step S2, forming a network traffic grayscale image set. The network traffic grayscale image set includes network traffic grayscale images corresponding to normal traffic and abnormal traffic. Abnormal traffic includes network traffic generated when DDoS and / or port scanning occur. Random data augmentation operations are performed on the grayscale images of network traffic in the set of grayscale images of network traffic. The data augmentation operations include horizontal flipping, ±10% translation and Gaussian noise injection. The randomly augmented set of grayscale images of network traffic is divided into training, validation, and test sets in a ratio of 7:2:

1. S302, set the optimizer to AdamW, and the initial learning rate to 1×10⁻⁶. -4 The weight decays to 1×10 -5 ; The loss function is set to binary cross-entropy loss, and the expression for the loss function is: Set the batch size to 128, the training period to 100, and the early stop mechanism to terminate training if the validation set loss does not decrease for 5 consecutive times. The learning rate scheduling uses a cosine annealing strategy with a period of 10 epochs and a minimum learning rate of 1×10. -6 ; S303, Initialize the model, input the grayscale image of network traffic in the training set, calculate layer by layer through the four feature extraction units in the feature extraction module, and output the anomaly probability; S304 uses a loss function to calculate the loss and update the parameters of the feature extraction unit, setting the gradient clipping threshold to 5.0 during the process; S305. After each epoch, the F1 score is evaluated on the validation set. After all epochs are completed, the traffic detection model with the best evaluation result is saved as the best model. S306. Load the best model, calculate the accuracy, recall, and AUC-ROC on the test set. If the manually set threshold is not met, repeat steps S303-S306 until the manually set threshold is met, and then stop to obtain the trained traffic detection model.

8. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the abnormal network traffic detection method as described in any one of claims 1-6.

9. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the abnormal network traffic detection method as described in any one of claims 1-6.

Images